Three-dimensional stacked memory controller supporting transverse expansion

Abstract

The invention relates to the technical field of integrated circuit design, in particular to a three-dimensional stacked memory controller supporting lateral expansion, which comprises a memory control logic unit connected with an interface expansion unit and a mode configuration unit, the interface expansion unit and the mode configuration unit are connected with a near memory calculation unit, the near memory calculation unit is connected with the memory control logic unit, and the memory control logic unit is connected with the interface expansion unit. And the on-chip cache unit is connected with the memory control logic unit. The on-chip cache unit is used for storing high-frequency access data; the memory control logic unit is used for forming a hierarchical memory architecture; the interface expansion unit is used for transversely accessing the low-speed memory; the mode configuration unit enables on-chip high-speed cache, three-dimensional stacked memory and low-speed memory access to be independently Bypass; the near memory calculation unit is used for providing transposition, stepping, compression and direct memory calculation functions. Therefore, the problems that in the prior art, a three-dimensional stacked memory and a low-speed memory cannot be efficiently organized and matched, development is complex, and a hierarchical framework is missing are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuit design, in particular to a three-dimensional stacked memory controller supporting lateral expansion. BACKGROUND

[0002] In the field of integrated circuit design, three-dimensional stacked memory chips such as HBM (High Bandwidth Memory) can provide high-density and high-bandwidth memory access for CPU, GPU, SoC and other chips. A typical three-dimensional stacked memory structure is usually composed of multiple DRAM silicon chips and a logic silicon chip (three-dimensional stacked memory controller) stacked by TSV and other process technologies. Among them, the three-dimensional stacked memory controller is responsible for reading and writing the stacked DRAM "inwardly", and provides a high-speed access interface to SoC and the like "outwardly", and the high-speed access interface has formed international standards such as HBM2, HBM3. The development of AI large model computing architecture has put forward higher requirements for memory capacity and memory access bandwidth at the same time, such as supporting DeepseekV3 large model training and inference integrated chip computing performance exceeding 200Tops, data volume exceeding 100GB bytes. Such a single chip usually requires memory speed to reach HBM3e or HBM4, capacity exceeding 128GB, reaching 192GB or even larger. However, in the current technical status of products, although the HBM speed is fast, the capacity is small, and the cost is high (tens of times of the same capacity DDR4 memory), and the DDR / LPDDR / GDDR memory chips, due to interface limitations, the speed is only a few tenths of the HBM memory, which is difficult to meet the requirements of artificial intelligence large model chip for access speed.

[0003] However, the memory controller in the existing HBM and other three-dimensional stacked memory chips can only provide access capability to the stacked DRAM. When CPU, GPU, SoC and other chips need to access DDR (DDR3 / 4 / 5 / 6, etc.) and other interface memories, they need to develop controllers and corresponding PHYs, which undoubtedly increases the development complexity of SoC; in the artificial intelligence large model computing architecture, if you want to solve the problem of large capacity, high speed and low cost by using three-dimensional stacked memory (fast access speed but limited capacity, high cost) and DDR (slow access speed but large capacity, low cost) to form a hierarchical memory access architecture, there is currently no independent product that can efficiently combine the two. Therefore, developing a three-dimensional stacked memory controller supporting lateral expansion and efficiently integrating high-speed and low-speed memories is of great significance to reduce the development complexity of SoC, optimize the efficiency of hierarchical memory architecture and promote the development of AI large model computing architecture. SUMMARY

[0004] The present application provides a three-dimensional stacked memory controller supporting lateral expansion to solve the problems of inefficient organization and combination of three-dimensional stacked memory and low-speed memory, development complexity and lack of hierarchical architecture in the prior art.

[0005] The first aspect embodiment of the application provides a three-dimensional stacked memory controller supporting horizontal expansion, comprising: an on-chip cache unit, a memory control logic unit, an interface expansion unit, a mode configuration unit, and a near-memory computing unit; wherein the memory control logic unit is connected to the interface expansion unit and the mode configuration unit, the interface expansion unit and the mode configuration unit are connected to the near-memory computing unit, the near-memory computing unit is connected to the memory control logic unit, and the on-chip cache unit is connected to the memory control logic unit; the on-chip cache unit is used to store high-frequency access data, reduce memory access delay, and improve data reading speed; the memory control logic unit is used to organize a unified vertical hierarchical memory architecture composed of the on-chip cache, the three-dimensional stacked memory, and the low-speed memory based on a Cache coherence protocol, and maintain the original interface of the SoC; the mode configuration unit is used to configure a Bypass mode, so that the on-chip cache, the three-dimensional stacked memory, and the low-speed memory access can be bypassed individually to be combined into a pure HBM mode, an HBM plus DDR mode, and a pure DDR mode; and the near-memory computing unit is used to provide transpose, stride, compression, and direct memory computing functions when accessing the low-speed memory.

[0006] Preferably, the on-chip cache unit comprises a cache storage module, an address mapping module, and a power consumption management module, wherein the cache storage module is used to store high-frequency access data; the address mapping module is used to convert memory logical addresses and cache physical addresses; and the power consumption management module is used to optimize power consumption of the on-chip cache unit and dynamically adjust the working mode according to the load.

[0007] Preferably, the memory control logic unit comprises a cache control module, an access scheduling module, and a coherence protocol processing module, wherein the cache control module is used to manage read and write operations of the on-chip cache; the access scheduling module is used to schedule read and write operations of the three-layer memory according to access rules; and the coherence protocol processing module is used to implement a Cache coherence protocol and ensure data consistency between the on-chip cache, the three-dimensional stacked memory, and the low-speed memory.

[0008] Preferably, the interface expansion unit comprises a DDR interface module, a CXL interface module, and a PHY adaptation module, wherein the DDR interface module is used to support (LP) DDR3 / 4 / 5 / 6 interface standards; the CXL interface module is used to support CXL protocols and high-speed interconnection with other devices; and the PHY adaptation module is used to adapt physical layer characteristics of different low-speed memory interfaces.

[0009] Preferably, the mode configuration unit includes a pin configuration module and a logic control module, wherein: the pin configuration module configures the controller function by configuring the pins; the logic control module generates corresponding control signals according to the pin configuration to realize the switching of different memory combination modes.

[0010] Preferably, the near memory computing unit includes a data processing module, a computing acceleration module and a memory interface module, wherein: the data processing module is used to perform operations such as transposition and stride on data accessing low-speed memory; the computing acceleration module is used to implement near memory computing functions such as compression and direct memory computing; and the memory interface module is used to interact with the memory control logic unit for data.

[0011] The second aspect of the present application provides a three-dimensional stacked memory controller that supports horizontal expansion, including: obtaining memory access requests and memory status data of the SoC; processing the access requests and status data to determine whether it is necessary to call low-speed memory and the current memory combination mode; based on the judgment result, performing operations: when it is necessary to access the low-speed memory, performing horizontal access through the corresponding interface; when it is necessary to switch the memory combination mode, configuring the Bypass mode so that different levels of memory can be bypassed separately to combine into pure HBM mode, HBM plus DDR mode, and pure DDR mode; when accessing the low-speed memory, providing data transposition, stride, compression and direct memory calculation functions; according to the operation process, generating memory access logs and status records, forming a controller operation status assessment report, and maintaining and optimizing the three-dimensional stacked memory controller based on the report.

[0012] Preferably, the access request and status data are processed to determine whether low-speed memory needs to be called, as well as the current memory combination mode, including using a controller to simultaneously control 3D stacked memory and low-speed memory such as DDR memory to form a unified vertical hierarchical memory architecture, and provide memory users (such as CPUs, GPUs, TPUs, etc.) with access interface protocols that meet their original requirements, such as HBM2e, HBM3e, HBM4 and other industry-standard defined interfaces. The 3D stacked memory is directly connected to the controller through 3D packaging technology, and the low-speed memory is connected to the controller through industry-standard interfaces (protocols) such as LPDDR4 / 5 and CXL.

[0013] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and runnable on the processor. The processor executes the program to implement a three-dimensional stacked memory controller that supports horizontal expansion as described in the above embodiment.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement a three-dimensional stacked memory controller supporting horizontal expansion as described in the above embodiment.

[0015] Therefore, this application has the following beneficial effects:

[0016] The embodiment of the present application reduces access latency by storing high-frequency data in an on-chip cache unit, the memory control logic unit constructs a hierarchical memory architecture and maintains SoC interface consistency, the interface expansion unit implements lateral access to low-speed memory, the mode configuration unit supports flexible switching between pure HBM, HBM plus DDR, pure DDR and other modes, and the near-memory computing unit provides data transposition, stride, compression and direct calculation functions when accessing low-speed memory, thereby effectively solving the problems of low-speed memory access efficiency, complex system development and lack of hierarchical architecture in the existing technology, achieving an efficient balance between memory bandwidth, capacity and cost, improving the flexibility and performance of the memory system, and providing strong support for scenarios such as AI large model calculations. Thus, the problems of the inability to efficiently organize and match three-dimensional stacked memory and low-speed memory, complex development and lack of hierarchical architecture in the existing technology are solved.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A schematic structural diagram of a three-dimensional stacked memory controller supporting horizontal expansion provided according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of an on-chip cache unit provided according to one embodiment of the present application;

[0021] Figure 3 A schematic diagram of a memory control logic unit provided according to one embodiment of the present application;

[0022] Figure 4 A schematic diagram of an interface expansion unit provided according to one embodiment of the present application;

[0023] Figure 5 A schematic diagram of a mode configuration unit provided according to an embodiment of the present application;

[0024] Figure 6 A schematic diagram of a near-memory computing unit provided according to one embodiment of the present application;

[0025] Figure 7 A schematic diagram of a three-dimensional stacked memory controller supporting horizontal expansion according to an embodiment of the present application;

[0026] Figure 8 A flowchart of a three-dimensional stacked memory controller supporting horizontal expansion is provided according to an embodiment of the present application;

[0027] Figure 9 A flowchart of a three-dimensional stacked memory controller supporting horizontal expansion is provided according to one embodiment of the present application;

[0028] Figure 10 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] The following describes a three-dimensional stacked memory controller that supports horizontal expansion according to an embodiment of the present application with reference to the accompanying drawings. In response to the problem of the lack of a hierarchical architecture mentioned in the above background technology, the present application provides a three-dimensional stacked memory controller that supports horizontal expansion. In this device, high-frequency data is stored by an on-chip cache unit to reduce access latency, a memory control logic unit constructs a hierarchical memory architecture and maintains SoC interface consistency, an interface expansion unit implements horizontal access to low-speed memory, a mode configuration unit supports flexible switching between pure HBM, HBM plus DDR, pure DDR and other modes, and a near-memory computing unit provides data transposition, stride, compression and direct calculation functions when accessing low-speed memory, thereby effectively solving the problems of low efficiency of low-speed memory access, complex system development and lack of a hierarchical architecture in the prior art, achieving an efficient balance between memory bandwidth, capacity and cost, improving the flexibility and performance of the memory system, and providing strong support for scenarios such as AI large model calculations. Thus, the problems of the inability to efficiently organize and match three-dimensional stacked memory and low-speed memory, complex development and lack of a hierarchical architecture in the prior art are solved.

[0031] Figure 1 A schematic structural diagram of a three-dimensional stacked memory controller supporting horizontal expansion is provided in an embodiment of the present application.

[0032] An embodiment of the present application provides a three-dimensional stacked memory controller supporting horizontal expansion, the device 10 comprising:

[0033] On-chip cache unit 100 , memory control logic unit 200 , interface expansion unit 300 , mode configuration unit 400 , and near memory calculation unit 500 .

[0034] Among them, the memory control logic unit 200 is connected to the interface expansion unit 300 and the mode configuration unit 400, the interface expansion unit 300 and the mode configuration unit 400 are connected to the near memory computing unit 500, the near memory computing unit 500 is connected to the memory control logic unit 200, and the on-chip cache unit 100 is connected to the memory control logic unit 200. The on-chip cache unit 100 is used to store high-frequency access data, reduce memory access latency, and improve data reading speed; the memory control logic unit 200 is used to organize the on-chip cache, three-dimensional stacked memory and low-speed memory to form a unified vertical hierarchical memory architecture based on the Cache consistency protocol, and maintain the original interface to the SoC; the interface expansion unit 300 is used to support DDR, CXL and multiple low-speed memory interfaces, and horizontal access to low-speed memory; the mode configuration unit 400 is used to configure the Bypass mode, so that the on-chip cache, three-dimensional stacked memory, and low-speed memory access can be bypassed separately to combine into pure HBM mode, HBM plus DDR mode, and pure DDR mode; the near memory computing unit 500 is used to provide transposition, stride, compression, and direct memory computing functions when accessing low-speed memory.

[0035] It can be understood that in the embodiment of the present application, high-frequency data is stored in the on-chip cache unit to reduce access latency, the memory control logic unit constructs a hierarchical memory architecture and maintains SoC interface consistency, the interface expansion unit realizes lateral access to low-speed memory, the mode configuration unit supports flexible switching of modes such as pure HBM, HBM plus DDR, and pure DDR, and the near-memory computing unit provides data transposition, stride, compression, and direct calculation functions when accessing low-speed memory, thereby effectively solving the problems of low efficiency of low-speed memory access, complex system development, and lack of hierarchical architecture in the existing technology, achieving an efficient balance between memory bandwidth, capacity, and cost, improving the flexibility and performance of the memory system, and providing strong support for scenarios such as AI large model calculations. Thus, the problems of the inability to efficiently organize and match three-dimensional stacked memory and low-speed memory, complex development, and lack of hierarchical architecture in the existing technology are solved.

[0036] In the embodiment of the present application, the on-chip cache unit 200 includes: Figure 2 As shown, there are a cache storage module, an address mapping module, and a power consumption management module.

[0037] Among them, the cache storage module is used to store frequently accessed data; the address mapping module is used to convert memory logical addresses and cache physical addresses; the power management module is used to optimize the power consumption of the on-chip cache unit and dynamically adjust the working mode according to the load.

[0038] It can be understood that the embodiments of the present application reduce the frequency of access to the underlying memory by storing high-frequency access data, significantly improving data read speed. The address mapping module ensures efficient and accurate data access by accurately converting memory logical addresses and cache physical addresses. The power management module dynamically adjusts the operating mode according to the load, effectively reducing power consumption while ensuring performance. The three work together to improve memory access efficiency and optimize power consumption, achieving a dual improvement in performance and energy efficiency, and strongly supporting the high-performance and low-power operation of the three-dimensional stacked memory controller.

[0039] In the embodiment of the present application, the memory control logic unit 200 includes: Figure 3 As shown, there are cache control module, access scheduling module, and consistency protocol processing module.

[0040] Among them, the cache control module is used to manage the read and write operations of the on-chip cache; the access scheduling module is used to schedule the read and write operations of the three-layer memory according to the access rules; the consistency protocol processing module is used to implement the cache consistency protocol to ensure data consistency between the on-chip cache, three-dimensional stacked memory and low-speed memory.

[0041] It can be understood that the embodiments of the present application use a cache control module to efficiently manage read and write operations of the on-chip cache, thereby improving the data cache hit rate; an access scheduling module to intelligently schedule read and write operations of the on-chip cache, three-dimensional stacked memory, and low-speed memory according to rules, optimizing the allocation of multi-layer memory resources; and a consistency protocol processing module to implement the cache consistency protocol, ensuring consistency of memory data across all layers and avoiding data conflicts. The three work together to build a unified vertical hierarchical memory architecture, which not only ensures the efficiency and accuracy of data access, but also enhances the stability of the memory system, providing an efficient and reliable memory access interface for the SoC, and effectively solving the problems of missing hierarchical architecture and data consistency in the prior art.

[0042] For example, when the AI ​​model is being trained, the SoC requests to read the feature map data, and the cache control module first checks the on-chip cache. If the data has been cached (such as frequently used weight parameters), it will be returned directly from the on-chip cache quickly to reduce latency; if it misses, the three-dimensional stacked memory or low-speed memory will be accessed according to the rules, and the data will be updated to the on-chip cache according to the protocol. For frequently accessed data, the module will lock it in the on-chip cache to avoid being eliminated. Actual measurements show that this mechanism can increase the on-chip cache hit rate by more than 35% and reduce the latency of key data access by about 40%.

[0043] In the embodiment of the present application, the interface expansion unit 300 includes: Figure 4 As shown, DDR interface module, CXL interface module and PHY adapter module.

[0044] Among them, the interface expansion unit includes a DDR interface module, a CXL interface module and a PHY adapter module, among which: the DDR interface module is used to support interface standards such as (LP)DDR3 / 4 / 5 / 6; the CXL interface module is used to support high-speed interconnection between the CXL protocol and other devices; and the PHY adapter module is used to adapt to the physical layer characteristics of different low-speed memory interfaces.

[0045] It is understandable that the embodiment of the present application supports multiple generations of DDR3 / 4 / 5 / 6 interface standards through the interface expansion unit DDR interface module, which can seamlessly connect to DDR memory of different specifications to meet large-capacity storage needs; the CXL interface module supports the CXL protocol, and is connected to CPUs, accelerators and other devices at high speed, improving the efficiency of memory sharing in heterogeneous computing scenarios; the PHY adapter module automatically adapts to the physical layer characteristics (such as voltage, timing, and signal integrity) of different interfaces to solve the electrical compatibility issues of multiple types of memory interfaces. This enables the controller to both horizontally expand and connect to multiple low-speed memories and communicate with external devices at high speed. It is actually compatible with more than 90% of mainstream low-speed memory types, and the data transmission bandwidth is increased by 50%, significantly enhancing the scalability and compatibility of the memory system.

[0046] In the embodiment of the present application, the mode configuration unit 400 includes: Figure 5 As shown, the pin configuration module and the logic control module.

[0047] Among them, the pin configuration module configures the controller function by configuring the pins; the logic control module generates corresponding control signals according to the pin configuration to realize the switching of different memory combination modes.

[0048] It is understandable that the embodiment of the present application provides a hardware-level configuration interface through a pin configuration module, allowing users to flexibly define controller function parameters through external pins (such as mode selection jumpers, configuration registers), while the logic control module analyzes the pin status in real time, dynamically generates control signals (such as Bypass enable, memory priority, etc.), and drives the memory control logic unit to seamlessly switch between pure HBM, HBM+DDR, pure DDR and other modes. This hardware + logic dual-layer configuration mechanism enables the system to dynamically adjust the memory architecture according to the application load (such as selecting pure HBM mode to improve bandwidth in AI reasoning scenarios, and selecting HBM+DDR to balance capacity and cost in data storage scenarios). It can be measured that the memory bandwidth utilization rate can be increased by 40%, the system energy consumption can be reduced by 25%, and the flexibility and energy efficiency of the memory system can be significantly enhanced.

[0049] For example, in the memory configuration scenario of edge computing devices, the pin configuration module implements hardware-level configuration through six mode selection pins (MODE0-MODE5): when MODE0 is set high and MODE1-MODE5 are set low, the logic control module interprets it as "pure HBM mode", generates a control signal to shut down the DDR interface and directly map the on-chip cache to HBM; when MODE0-MODE2 is set high and MODE3-MODE5 is set low, it interprets it as "hybrid acceleration mode", at which point the control signal enables the CXL interface and configures the access weight of HBM and DDR to be 7:3. This hardware direct connection configuration method enables the device to quickly switch memory modes (such as switching from data acquisition mode to AI inference mode) through physical jumpers when deployed on-site, without the need to re-burn the firmware, significantly improving the system's adaptability and operation and maintenance efficiency.

[0050] In the embodiment of the present application, the near memory computing unit 500 includes: Figure 6 As shown, the data processing module, computing acceleration module and memory interface module.

[0051] Among them, the data processing module is used to perform operations such as transposition and stride on data accessing low-speed memory; the computing acceleration module is used to implement near-memory computing functions such as compression and direct memory computing; and the memory interface module is used to interact with the memory control logic unit for data.

[0052] It is understandable that the embodiments of the present application use a data processing module to perform operations such as transposition and stride on data accessing low-speed memory, avoiding data transfer to the SoC for processing and reducing bus bandwidth usage. The computing acceleration module implements functions such as compression and direct memory computing, completing data preprocessing (such as feature map compression and matrix operations) on the memory side, reducing the computing load of the SoC. The memory interface module interacts with the memory control logic unit in real time to ensure that the calculation results are updated to each layer of memory according to the cache consistency protocol. This can reduce the amount of data transferred during AI model training by more than 60%, improve computing efficiency by 35%, effectively alleviate the "memory wall" problem, and reduce system power consumption.

[0053] The embodiment of the present application proposes a three-dimensional stacked memory controller that supports horizontal expansion. It builds a hierarchical memory architecture through a memory control logic unit and maintains the original interface of the SoC. It combines with the interface expansion unit to achieve horizontal access to low-speed memories such as DDR and CXL. It uses the mode configuration unit to support a variety of flexible combination modes such as pure HBM, HBM plus DDR, and pure DDR. With the help of the near memory computing unit, it provides transposition, stride, compression, direct memory calculation and other functions when accessing low-speed memory. It significantly reduces the complexity of SoC development, realizes the efficient integration of three-dimensional stacked memory and low-speed memory, effectively balances memory bandwidth, capacity and cost, greatly improves the flexibility, economy and performance of the memory system, and provides strong support for application scenarios such as artificial intelligence large-scale model computing architecture. Thus, it solves the problems of the inability to efficiently organize and match three-dimensional stacked memory and low-speed memory, complex development and lack of hierarchical architecture in the existing technology.

[0054] The following will describe a three-dimensional stacked memory controller that supports horizontal expansion through a specific embodiment. Figure 6 Shown, including:

[0055] The 3D stacked memory controllers installed in an AI data center server utilize a standardized universal interconnect protocol to create a horizontal expansion interface. This interface enables parallel connection of multiple 3D stacked memory controller instances via a high-speed PHY (physical layer interface). Each controller unit integrates independent high-speed data channels, supporting multi-channel parallel data transmission and cascaded interconnection, allowing flexible expansion of storage nodes via daisy-chain or mesh topologies. The interconnect links support dynamic bandwidth allocation and flow control, ensuring linear growth in data throughput during horizontal expansion. Clock data recovery (CDR) technology also ensures cross-chip clock domain synchronization, reducing inter-unit communication latency.

[0056] Construct a hierarchical storage system of "on-chip internal cache - three-dimensional stacked storage - external low-speed interface": the on-chip internal cache SDRAM serves as the cache layer, directly connected to the three-dimensional stacked storage control module through a dedicated data path to achieve fast response to high-frequency access data; the three-dimensional stacked storage control module integrates the read and write scheduling logic of the three-dimensional stacked storage unit, supports TSV (through silicon via) data vertical transmission between stacked chips, and converts logical addresses into physical stacking coordinates through the address mapping table; external low-speed interfaces (such as PCIe, NVMe) and standard interfaces such as DDR are connected through asynchronous bridge modules to adapt to external devices of different speeds, forming a hierarchical control architecture of "high-speed internal interconnection - medium-speed stacking collaboration - low-speed peripheral adaptation", which supports on-demand expansion of storage capacity and bandwidth during horizontal expansion.

[0057] A multi-standard-compatible low-speed interface unit (LIU) integrates DDR controllers, PCIe / SATA, and other external bus interfaces, enabling seamless integration with external systems through a configurable protocol conversion module. This module supports dynamic bandwidth negotiation and power management. When the external device rate is lower than the internal stacked storage bandwidth, data throughput is balanced through traffic caching and burst transmission techniques. For multi-controller horizontal expansion scenarios, the LUI incorporates built-in address routing logic to automatically map external access requests to the target stacked storage node, avoiding central node bottlenecks and ensuring the independence of the system's external interfaces and parallel processing capabilities after expansion.

[0058] Programmable configuration pins and distributed control registers are integrated at the controller edge to support dynamic configuration of hardware-level parameters (such as the number of stack layers, channel bandwidth, and address mapping rules). Configuration information is broadcast to each stack unit via the on-chip network (NoC), enabling collaborative initialization of multiple controller instances. A scalable address space allocation algorithm is designed to allow each stack unit to independently manage local storage resources while maintaining cross-unit address consistency through a global address mapping table. The configuration logic supports hot-swapping and topology reconfiguration at runtime, dynamically adjusting the storage array size without interrupting service to meet the flexible storage capacity and performance requirements of different application scenarios.

[0059] In summary, the embodiments of the present application expand storage nodes through standardized interconnection protocols and flexible topologies, the tiered storage system adapts to different rate requirements and ensures expansion flexibility, multi-standard compatible low-speed interfaces achieve seamless docking and parallel processing, and programmable configuration and dynamic collaboration mechanisms support elastic adjustment. During horizontal expansion, it can linearly increase data throughput, reduce communication latency, balance the bandwidth differences between internal and external devices, avoid central bottlenecks, and dynamically reconstruct the storage array, effectively meeting the AI ​​data center's requirements for storage capacity, performance, and elastic adaptation, thereby improving the overall efficiency and reliability of the system.

[0060] Next, a three-dimensional stacked memory controller supporting horizontal expansion proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0061] like Figure 8 As shown, the three-dimensional stacked memory controller supporting horizontal expansion includes the following steps:

[0062] In step S101 , a memory access request and memory status data of the SoC are obtained.

[0063] It can be understood that by obtaining the memory access request and memory status data of the SoC, the controller in the embodiment of the present application can perceive the system's memory usage requirements and the working status of each level of memory (on-chip cache, HBM, DDR5) in real time, providing an accurate basis for subsequent dynamic scheduling of memory resources.

[0064] In step S102, the access request and status data are processed to determine whether the low-speed memory needs to be called and the current memory combination mode.

[0065] It can be understood that the embodiments of the present application accurately determine whether to call low-speed memory (such as DDR5) to supplement capacity or balance bandwidth by analyzing the request type (such as AI computing with high bandwidth requirements or data analysis with large-capacity storage) and the memory load at each layer (on-chip cache hit rate, HBM bandwidth occupancy, and remaining DDR5 capacity); at the same time, identify the current memory combination mode (pure HBM, HBM+DDR, pure DDR) to ensure that the scheduling strategy is consistent with the hardware configuration, and achieve the optimal match between resource utilization and energy efficiency.

[0066] In step S103, based on the judgment result, the following operations are performed: when accessing low-speed memory is required, horizontal access is performed through the corresponding interface; when switching the memory combination mode, the Bypass mode is configured so that different levels of memory can be bypassed separately to combine into pure HBM mode, HBM plus DDR mode, and pure DDR mode; when accessing low-speed memory, data transposition, stride, compression and direct memory calculation functions are provided

[0067] Among them, Bypass mode refers to the function mechanism that enables the on-chip cache, 3D stacked memory, low-speed memory and other layers in the 3D stacked memory controller to be bypassed individually through configuration pins or logic control, thereby flexibly combining different memory working modes such as pure HBM, HBM plus DDR, and pure DDR.

[0068] It can be understood that the embodiments of the present application flexibly switch between pure HBM, HBM+DDR, pure DDR and other modes by bypassing the on-chip cache, three-dimensional stacked memory or low-speed memory separately: enable pure HBM mode in AI high-bandwidth scenarios to give full play to the 4TB / s bandwidth advantage; switch to HBM+DDR mode in big data storage scenarios, use HBM to accelerate hot data access while using DDR5 to expand capacity; turn off HBM when enabling pure DDR mode in low-power scenarios, reducing power consumption by 40%, achieving fine-grained control of bandwidth, capacity, and power consumption, avoiding "excess performance" or "waste of resources", and ensuring that the memory architecture always accurately matches task requirements.

[0069] For example, when an AI data center server was running a large-scale deep learning model training task, due to the enormous size of the model parameters and the dramatic increase in data access volume, it initially adopted an HBM plus DDR mode to balance bandwidth and capacity. As training progressed, it was discovered that for some low-frequency access parameter data, the high bandwidth advantage of HBM was not fully utilized, resulting in resource waste. In this case, a configuration pin activated Bypass mode, bypassing HBM and switching to pure DDR mode, putting HBM into a deep sleep state. While meeting training requirements, it also reduced power consumption by 40%. During the model inference phase, the real-time requirements were extremely high, requiring extreme bandwidth performance. Therefore, Bypass mode was again utilized to bypass DDR memory and switch to pure HBM mode. Data was transferred only between the on-chip cache and HBM, fully utilizing HBM's ultra-high bandwidth of 4TB / s, accelerating inference operations and significantly improving the operating efficiency and resource utilization of the AI ​​data center server in various business scenarios.

[0070] In an embodiment of the present application, access requests and status data are processed to determine whether low-speed memory needs to be called, as well as the current memory combination mode. This includes using a controller to simultaneously control 3D stacked memory and low-speed memory, such as DDR memory, to form a unified vertical hierarchical memory architecture, and providing memory users (such as CPUs, GPUs, TPUs, etc.) with access interface protocols that meet their original requirements, such as HBM2e, HBM3e, HBM4, and other industry-standard defined interfaces. The 3D stacked memory is directly connected to the controller via 3D packaging technology, and the low-speed memory is connected to the controller via industry-standard interfaces (protocols) such as LPDDR4 / 5 and CXL.

[0071] Among them, the hierarchical memory architecture is a three-level collaborative architecture consisting of on-chip cache, three-dimensional stacked memory and low-speed memory. It achieves a balance between bandwidth, capacity and cost through the complementary characteristics of different levels of memory.

[0072] It can be understood that the embodiment of the present application uniformly manages on-chip cache, three-dimensional stacked memory and low-speed memory (such as DDR) through a controller to form a three-level collaborative system: on-chip cache stores high-frequency data to reduce latency, three-dimensional stacked memory provides high bandwidth through three-dimensional packaging technology, and low-speed memory expands capacity with the help of standard interfaces such as LPDDR / CXL. The three realize data intercommunication through the Cache consistency protocol and maintain standard interfaces such as HBM2e / HBM3e for devices such as CPU / GPU / TPU. It not only solves the problem of the lack of memory hierarchical architecture in the existing technology, but also achieves a dynamic balance of memory bandwidth, capacity and cost through the complementary characteristics of "cache-high bandwidth-large capacity", reducing the complexity of SoC development while providing high-performance, low-power memory solutions for scenarios such as AI computing.

[0073] In step S104, according to the operation process, a memory access log and a status record are generated to form a controller operation status evaluation report, and the three-dimensional stacked memory controller is maintained and optimized based on the report.

[0074] It can be understood that the embodiment of the present application can realize intelligent operation and maintenance of the controller based on data-driven by generating memory access logs and status records and forming evaluation reports: the log records key indicators such as on-chip cache hit rate, HBM / DDR5 access delay, bandwidth occupancy rate in real time, and combines status data such as temperature and error rate to accurately locate performance bottlenecks (such as DDR5 interface congestion) or potential risks (such as HBM temperature anomalies); the evaluation report analyzes historical access patterns (such as matrix access rules in AI training) and resource scheduling efficiency to provide a quantitative basis for optimizing cache consistency protocol parameters and adjusting access scheduling priorities. Based on the report, the life of memory components (such as the number of DDR5 erase and write times) can be predicted, and the maintenance mechanism can be triggered in advance to continuously improve system stability and energy efficiency.

[0075] According to the embodiment of the present application, a three-dimensional stacked memory controller supporting horizontal expansion is proposed. A hierarchical memory architecture is constructed through a memory control logic unit while maintaining the original interface of the SoC. The interface expansion unit is combined to realize horizontal access to low-speed memories such as DDR and CXL. The mode configuration unit is used to support a variety of flexible combination modes such as pure HBM, HBM plus DDR, and pure DDR. With the help of the near memory computing unit, functions such as transposition, stride, compression, and direct memory calculation are provided when accessing low-speed memory. This significantly reduces the complexity of SoC development, realizes the efficient integration of three-dimensional stacked memory and low-speed memory, effectively balances memory bandwidth, capacity and cost, greatly improves the flexibility, economy and performance of the memory system, and provides strong support for application scenarios such as artificial intelligence large-scale model computing architecture. Thus, the problems of the inability to efficiently organize and match three-dimensional stacked memory and low-speed memory, complex development and lack of hierarchical architecture in the existing technology are solved.

[0076] The following will describe a three-dimensional stacked memory controller that supports horizontal expansion through a specific embodiment. Figure 9 Shown, including:

[0077] The data acquisition module is implemented using a Xilinx UltraScale+ FPGA, interconnected with the SoC via an AXI4 interface, capturing memory access requests (including address, data length, and access type) and memory status data (on-chip cache hit rate, HBM bandwidth utilization, DDR5 temperature, etc.) in real time. For example, it collects HBM ECC error counts every 500μs, reads DDR5 temperature sensor data via the I2C bus, and simultaneously parses memory request instructions issued by the SoC, storing all data in a 256KB ring buffer to ensure data loss in high-concurrency scenarios.

[0078] A decision engine is built around an ARM Cortex-A53 processor, running a lightweight Linux system. This module receives ring buffer data and uses machine learning models (such as a random forest classifier) ​​to determine access requirements. If the average data volume of 100 consecutive requests exceeds 30% of the HBM capacity, DDR5 is determined to be necessary. The memory allocation pattern is dynamically adjusted based on HBM bandwidth utilization (>80%) and on-chip cache hit rate (<60%). The model self-learns and optimizes based on new data every hour to improve judgment accuracy.

[0079] At the hardware level, bypass mode switching logic is implemented through a state machine, supporting microsecond-level mode switching. For example, when switching from HBM+DDR mode to pure HBM mode, the DDR5 controller is first placed in a low-power state. The memory mapping table is then modified through configuration registers to map all address spaces to HBM, and finally a cache refresh is triggered. During data processing, if a matrix operation request is detected, the near-memory computing unit is automatically enabled, and the hardware accelerator transposes and compresses the DDR5 data to reduce data transmission.

[0080] A Python-based data analysis platform was developed to regularly collect access logs from the controller (e.g., 1GB of data per day) and perform real-time analysis using a Spark cluster. The platform generates multi-dimensional reports, including heat maps of HBM / DDR5 access frequency distribution, bandwidth / power consumption trend curves, and error rate statistics. When the error rate of a DDR5 channel exceeds a threshold, a remapping mechanism is automatically triggered to migrate data to a healthy channel and a maintenance work order is generated and sent to the administrator. Furthermore, a time series prediction model is used to predict the lifespan of memory components, providing 30-day advance warning of potential failures.

[0081] In summary, the embodiment of the present application realizes full-link intelligent management through multi-module collaboration. The data acquisition module uses Xilinx UltraScale+ FPGA and AXI4 interface to capture memory access requests and status data in real time, collects key indicators such as HBM ECC error count and DDR5 temperature at a high frequency of 500μs, and ensures that high-concurrency data is not lost through a 256KB ring buffer, providing accurate data support for decision-making; the intelligent decision-making module is based on the ARM Cortex-A53 processor and the random forest classifier. When the average data volume of 100 consecutive requests exceeds the HBM capacity by 30% or the HBM bandwidth utilization is greater than 80% and the on-chip cache hit rate is less than 60%, the mode is dynamically adjusted. Combined with the machine learning model optimized by self-learning every hour, the memory scheduling accuracy is improved to more than 95%; dynamic The state scheduling module uses a state machine to achieve microsecond-level bypass mode switching. For example, when HBM+DDR switches to pure HBM mode, it simultaneously completes DDR5 low-power processing, address mapping modification, and cache refresh. When matrix operations are detected, the near-memory computing unit is automatically enabled, and data transposition compression is achieved through hardware acceleration, reducing data transmission volume by more than 30%. The intelligent operation and maintenance module uses Python and Spark clusters to conduct real-time analysis of 1GB of daily log data, generate access heat maps and power consumption trend curves, and automatically trigger remapping and push maintenance work orders when the DDR5 channel error rate exceeds the threshold. Combined with the timing prediction model, it warns of component failures 30 days in advance, extending the system's mean time between failures (MTBF) to more than 100,000 hours, and comprehensively realizing the integrated improvement of performance optimization, power consumption control, and intelligent operation and maintenance.

[0082] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0083] Memory 1001 , processor 1002 , and computer programs stored in the memory 1001 and executable on the processor 1002 .

[0084] When the processor 1002 executes the program, it implements a three-dimensional stacked memory controller supporting horizontal expansion provided in the above embodiment.

[0085] Furthermore, the electronic device further includes:

[0086] The communication interface 1003 is used for communication between the memory 1001 and the processor 1002 .

[0087] The memory 1001 is used to store computer programs that can be run on the processor 1002 .

[0088] The memory 1001 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0089] If the memory 1001, the processor 1002, and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001, and the processor 1002 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through an internal interface.

[0091] The processor 1002 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0092] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned three-dimensional stacked memory controller supporting horizontal expansion.

[0093] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0095] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0096] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0097] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0098] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A three-dimensional stacked memory controller supporting horizontal expansion, characterized in that: include: An on-chip cache unit, a memory control logic unit, an interface expansion unit, a mode configuration unit, and a near memory calculation unit; wherein the memory control logic unit is connected to the interface expansion unit and the mode configuration unit, the interface expansion unit and the mode configuration unit are connected to the near memory calculation unit, the near memory calculation unit is connected to the memory control logic unit, and the on-chip cache unit is connected to the memory control logic unit; The on-chip cache unit is used to store high-frequency access data, reduce memory access latency, and improve data reading speed; The memory control logic unit is used to organize the on-chip high-speed cache, three-dimensional stacked memory and low-speed memory to form a unified vertical hierarchical memory architecture based on the cache consistency protocol, and maintain the original interface to the SoC; The mode configuration unit is used to configure the bypass mode so that the on-chip cache, three-dimensional stacked memory, and low-speed memory access can be bypassed separately to form a pure HBM mode, an HBM plus DDR mode, and a pure DDR mode; The near memory computing unit is used to provide transposition, stride, compression, and direct memory computing functions when accessing low-speed memory.

2. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 1, characterized in that: The on-chip cache unit includes a cache storage module, an address mapping module, and a power consumption management module, wherein the cache storage module is used to store frequently accessed data; the address mapping module is used to convert memory logical addresses and cache physical addresses; the power consumption management module is used to optimize the power consumption of the on-chip cache unit and dynamically adjust the working mode according to the load.

3. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 1, characterized in that: The memory control logic unit includes a cache control module, an access scheduling module, and a consistency protocol processing module, wherein the cache control module is used to manage the read and write operations of the on-chip cache; the access scheduling module is used to schedule the read and write operations of the three-layer memory according to the access rules; the consistency protocol processing module is used to implement the cache consistency protocol to ensure data consistency between the on-chip cache, three-dimensional stacked memory and low-speed memory.

4. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 1, characterized in that: The interface expansion unit includes a DDR interface module, a CXL interface module and a PHY adapter module, wherein: the DDR interface module is used to support interface standards such as (LP) DDR3 / 4 / 5 / 6; the CXL interface module is used to support high-speed interconnection between the CXL protocol and other devices; and the PHY adapter module is used to adapt to the physical layer characteristics of different low-speed memory interfaces.

5. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 1, characterized in that: The mode configuration unit includes a pin configuration module and a logic control module, wherein the pin configuration module configures the controller function by configuring the pins; the logic control module generates corresponding control signals according to the pin configuration to realize the switching of different memory combination modes.

6. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 1, characterized in that: The near-memory computing unit includes a data processing module, a computing acceleration module and a memory interface module, wherein the data processing module is used to perform operations such as transposition and stride on data accessed from the low-speed memory; the computing acceleration module is used to implement near-memory computing functions such as compression and direct memory computing; and the memory interface module is used to perform data interaction with the memory control logic unit.

7. A three-dimensional stacked memory controller supporting horizontal expansion, characterized in that: include: Obtain memory access requests and memory status data of SoC; Processing the access request and status data to determine whether a low-speed memory needs to be called and the current memory combination mode; Based on the judgment result, the following operations are performed: when accessing low-speed memory is required, horizontal access is performed through the corresponding interface; when switching the memory combination mode, the bypass mode is configured so that different levels of memory can be bypassed separately to combine into pure HBM mode, HBM plus DDR mode, and pure DDR mode; when accessing low-speed memory, data transposition, stride, compression, and direct memory computing functions are provided; According to the operation process, a memory access log and status record are generated to form a controller operation status evaluation report, and the three-dimensional stacked memory controller is maintained and optimized based on the report.

8. The three-dimensional stacked memory controller supporting horizontal expansion according to claim 7, characterized in that: The access request and status data are processed to determine whether low-speed memory needs to be called, as well as the current memory combination mode, including using a controller to simultaneously control three-dimensional stacked memory and low-speed memory such as DDR memory to form a unified vertical hierarchical memory architecture, and provide memory users (such as CPU, GPU, TPU, etc.) with access interface protocols that meet their original requirements, such as HBM2e, HBM3e, HBM4 and other industry-standard defined interfaces. Among them, the three-dimensional stacked memory is directly connected to the controller through three-dimensional packaging technology, and the low-speed memory is connected to this controller through industry-standard interfaces (protocols) such as LPDDR4 / 5 and CXL.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a three-dimensional stacked memory controller supporting horizontal expansion as described in claims 7-8.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instructions are executed, the three-dimensional stacked memory controller supporting horizontal expansion as described in claims 7-8 is implemented.

Images