Packaging structure and data communication method for multi-stack high-bandwidth memory

Abstract

The embodiment of the invention provides a packaging structure of a multi-stack high-bandwidth memory and a data communication method. The packaging structure comprises a multi-layer internet, at least two high-bandwidth memory stacks and a path selection control unit matched with each high-bandwidth memory stack, the multi-layer internet is arranged between the at least two high-bandwidth memory stacks, and the path selection control unit is connected with the multi-layer internet; each high-bandwidth memory stack comprises a plurality of DRAM (Dynamic Random Access Memory) bare chips and a bottom logic bare chip which are vertically stacked; the multi-layer internet network comprises at least two transverse interconnection communication paths and is used for realizing data communication between the at least two high-bandwidth memory stacks; and the path selection control unit is configured to respond to a cross-stack target data access request, and select one target path from the at least two transverse interconnection communication paths according to a preset selection strategy to carry out target data transmission. By utilizing the packaging structure disclosed by the invention, the memory access flexibility and the parallel processing efficiency can be remarkably improved.

Description

Technical Field

[0001] This disclosure relates to the field of integrated circuit chip technology, specifically to a packaging structure and data communication method for multi-stack high-bandwidth memory. Background Technology

[0002] High-bandwidth memory (HBM) technology is a key solution to address memory bandwidth bottlenecks in high-performance computing, artificial intelligence acceleration, and other fields. It vertically stacks multiple DRAM dies using through-silicon vias (TSVs) and integrates them with main processors such as GPUs and CPUs within the same package using advanced 2.5D / 3D packaging technologies such as silicon interposers or embedded bridge dies, thus providing significantly greater bit width and capacity than traditional memory. However, existing standard HBM architectures have a significant drawback: multiple HBM stacks are typically configured as independent modules, with each stack communicating with the main processor (such as a GPU) only through its dedicated channel. When a computational task needs to access data located in another HBM stack, all data must first be moved to the main processor via interconnect structures such as silicon interposers, and then relayed by the main processor to the requesting HBM stack. This redundant data transport path not only introduces additional access latency but also consumes valuable main processor interconnect bandwidth, becoming a serious performance bottleneck in advanced computing paradigms such as high-concurrency data parallelism and model parallelism. Currently, the industry lacks an effective and universal solution that can establish an efficient and flexible direct communication mechanism between multiple HBM stacks within a single package. Summary of the Invention

[0003] To address the aforementioned technical problems, the present disclosure provides a solution. Embodiments of this disclosure offer a packaging structure and data communication method for multi-stack high-bandwidth memory.

[0004] According to a first aspect of the present disclosure, a packaging structure for multi-stack high-bandwidth memory is provided, wherein the packaging structure includes a multi-layer interconnect network, at least two high-bandwidth memory stacks, and a path selection control unit matching each of the high-bandwidth memory stacks, the multi-layer interconnect network being disposed between the at least two high-bandwidth memory stacks, and the path selection control unit being connected to the multi-layer interconnect network; Each of the high-bandwidth memory stacks includes multiple vertically stacked DRAM dies and a bottom logic die; The multi-layer interconnection network includes at least two horizontal interconnection communication paths for data communication between the at least two high-bandwidth memory stacks. The path selection control unit is configured to select a target path from the at least two lateral interconnect communication paths for target data transmission in response to a cross-stack target data access request, according to a predetermined selection strategy.

[0005] According to a second aspect of the present disclosure, a data communication method for an encapsulation structure is provided, wherein the encapsulation structure is the multi-stack high-bandwidth memory encapsulation structure described in the present disclosure, and the data communication method includes: The path selection control unit receives a target data access request for the target data initiated by the local computing module; wherein, the local computing module is integrated on the bottom logic die of the local high-bandwidth memory stack in the package structure; In response to the first path selection control unit determining that the target data is located in a remote high-bandwidth memory stack based on the target data access request, the control unit selects a target path from at least two lateral interconnect communication paths according to a predetermined selection strategy, and sends a data request instruction for the target data to the first lightweight cache controller that matches the local high-bandwidth memory stack in the target path. The first lightweight cache controller sends the data request instruction to the second lightweight cache controller that matches the remote high-bandwidth memory stack in the target path; In response to receiving the data request instruction, the second lightweight cache controller obtains the target data corresponding to the data request instruction from the remote high-bandwidth memory stack; The second lightweight cache controller uses the target path to feed the target data back to the first lightweight cache controller.

[0006] As described above, the packaging structure for multi-stack high-bandwidth memory provided in this disclosure integrates a distributed path selection control unit in the bottom logic die of each high-bandwidth memory stack and constructs a multi-dimensional communication network including bottom, middle, and top interconnect layers. This enables the local path selection control unit to intelligently select the optimal path based on data characteristics (such as latency requirements, data volume, and priority) when a stack needs to access remote data. It also coordinates with the lightweight cache controller (LCC) matching the local stack and the target stack on the optimal path to directly establish a data transmission channel between the stacks, thereby achieving direct memory access bypassing the main processor. This architecture effectively eliminates the latency bottleneck and bandwidth occupation caused by the main processor in traditional architectures, significantly improving the memory access flexibility, parallel processing efficiency, and overall system energy efficiency within the packaging structure. Attached Figure Description

[0007] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0008] Figure 1 This is a schematic diagram of a packaging structure for multi-stack high-bandwidth memory provided in an exemplary embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a lightweight cache controller (LCC) provided in an exemplary embodiment of this disclosure; Figure 3 This is a flowchart illustrating a data communication method for an encapsulation structure provided in an exemplary embodiment of this disclosure; Figure 4 This is a schematic diagram of the signal flow when cross-stack data communication is performed within the encapsulation structure provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0009] The present disclosure will be further described below with reference to the embodiments shown in the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0010] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0011] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0012] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0013] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0014] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0015] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0016] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0017] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0018] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0020] Overview of the inventive concept The core inventive concept of this disclosed technical solution lies in proposing an innovative multi-stack high-bandwidth memory package interconnect architecture. By setting a local path selection control unit in each high-bandwidth memory stack and constructing a horizontal interconnect communication path network including at least two physical layers (bottom, middle, and top), any high-bandwidth memory stack can intelligently select the optimal path based on the characteristics of the target data access request when it needs to access data from a remote stack. Then, based on the lightweight cache controller (LCC) matching the local stack and the remote (target) stack on the optimal path, a data transmission channel is directly established between the stacks, thereby eliminating the traditional step of relaying through the main processor, achieving a reduction in memory access latency and an improvement in overall system bandwidth efficiency.

[0021] Based on the above inventive concept, this disclosure proposes a packaging structure for multi-stack high-bandwidth memory and a data communication method for the packaging structure, as described in the following embodiments.

[0022] Example 1 This embodiment provides a packaging structure for multi-stack high-bandwidth memory. The packaging structure includes a multi-layer interconnect network, at least two high-bandwidth memory stacks, and a path selection control unit matching each of the high-bandwidth memory stacks. The multi-layer interconnect network is disposed between the at least two high-bandwidth memory stacks, and the path selection control unit is connected to the multi-layer interconnect network.

[0023] Each of the high-bandwidth memory stacks comprises multiple vertically stacked DRAM dies and a bottom logic die.

[0024] The multi-layer interconnection network includes at least two horizontal interconnection communication paths for data communication between the at least two high-bandwidth memory stacks.

[0025] The path selection control unit is configured to select a target path from the at least two lateral interconnect communication paths for target data transmission in response to a cross-stack target data access request, according to a predetermined selection strategy.

[0026] As an optional implementation method, refer to Figure 1 The package structure (i.e., the package) of the multi-stack high-bandwidth memory (HBM) shown primarily includes at least two HBM stacks arranged side-by-side (e.g., stack A and stack B), a multi-layer interconnect network, and a path selection control unit matching the number of stacks. Each HBM stack consists of multiple vertically stacked DRAM dies. Figure 1 The example shows three layers and a bottom logic die. The multi-layer interconnect network is positioned between stack A and stack B, providing at least two lateral interconnect communication paths at different physical levels. Each HBM stack's bottom logic die integrates a path selection control unit (i.e., the master control logic unit). Figure 1 The path selection control unit (hereinafter referred to as "main controller") is connected to the multi-layer interconnection network.

[0027] When a computing unit (module) integrated in stack A needs to access data located in stack B, the path selection control unit of stack A will respond to this cross-stack access request and select a target path from the available paths (i.e., at least two horizontal interconnect communication paths located at different physical levels) according to a preset strategy to establish a direct data channel between stack A and stack B and complete the data transmission.

[0028] As an alternative implementation, the at least two lateral interconnect communication paths include at least two selected from the following groups: bottom interconnect layer (BIL), middle interconnect layer (MIL), and top interconnect layer (TIL).

[0029] Reference Figure 1 For example, the BIL is located at the bottom of the stack, connecting the bottom logic dies of the two stacks; the MIL is embedded between the middle DRAM die layers of the two stacks (e.g., connecting the second DRAM die layer); and the TIL is located at the top of the two stacks. This example demonstrates a structure where BIL, MIL, and TIL coexist.

[0030] As an optional implementation, the bottom interconnect layer (BIL) is disposed at the bottom of the at least two high-bandwidth memory stacks, and the interconnection between the bottom logic dies of the at least two high-bandwidth memory stacks is realized through a bottom horizontal interconnect component.

[0031] The bottom horizontal interconnect component includes either a silicon interposer or an embedded silicon bridge.

[0032] Reference Figure 1 For example, a bottom interconnect layer (BIL) is located at the bottom of two HBM stacks, and interconnection between the two bottom logic dies is achieved through a bottom lateral interconnect component. This bottom lateral interconnect component can be a mature silicon interposer, utilizing high-density wiring on it for interconnection; or it can be implemented using advanced packaging technologies such as embedded silicon bridges (e.g., EMIB).

[0033] As an optional implementation, the Middle Interconnect Layer (MIL) is embedded between the middle DRAM die layers of the at least two high-bandwidth memory stacks and interconnected via laterally arranged bridging components. At least two lightweight cache controllers, matching the number of the at least two high-bandwidth memory stacks, are disposed along the path of the Middle Interconnect Layer. The correspondence between the lightweight cache controllers and the high-bandwidth memory stacks is one-to-one. The lightweight cache controller matching any one of the high-bandwidth memory stacks manages the transmission of target data on one side of that high-bandwidth memory stack. The bridging components include any one of bridging dies with microbumps, optical interconnect CPO modules, and micro SoIC bridges.

[0034] As one specific implementation method, refer to Figure 1 The Middle Interconnect Layer (MIL) is physically embedded between the DRAM die layers at the middle height of two High Bandwidth Memory (HBM) stacks. This interconnect is not achieved through a traditional packaging substrate or silicon interposer, but rather by directly connecting the corresponding die layers of the two stacks via a laterally positioned dedicated bridging component. This bridging component can be implemented using any of the following forms: bridging die + microbumps (e.g., using "bridging die + microbumps" to achieve high-density interconnection with the dies of the two stacks), optical interconnect CPO (co-packaged optics) modules, or miniature SoIC (system-on-a-chip) bridges, aiming to provide an extremely short, low-latency physical path.

[0035] Along this MIL path, a lightweight cache controller (LCC) is integrated to match the number of interconnected HBM stacks. Specifically, when two HBM stacks (e.g., stack A and stack B) are interconnected via MIL, two LCCs, LCC-A2 and LCC-B2, are configured within the MIL's bridging component. These two controllers have a one-to-one mapping to the two HBM stacks: LCC-A2 corresponds to and manages all target data transfers on the HBM stack A side of this path; similarly, LCC-B2 corresponds to and manages data transfers on the HBM stack B side. Optionally, in the middle interconnect layer (MIL), each LCC can be a highly optimized lightweight DMA-like controller with a streamlined design and focused functionality, whose core responsibility is to perform data block migration operations between its own HBM stack and the peer stack.

[0036] During operation, when the master control unit within the bottom logic die of an HBM stack (such as stack A) determines that a data access needs to be performed through the MIL path, it issues a start command to the LCC-A2 managing that side. This command includes key parameters such as source / destination addresses and data volume. After being triggered, LCC-A2 immediately interacts with the LCC-B2 on the opposite side via the MIL physical link, collaboratively establishing a point-to-point direct data channel. Subsequently, LCC-A2 is responsible for reading data from the HBM stack A it manages, or writing data from the other end to it, without the need for intervention from the upper-level general-purpose processor, thus completing the cross-stack data transfer task efficiently and with low latency.

[0037] As an optional implementation, the top interconnect layer is disposed on top of the at least two high-bandwidth memory stacks and interconnected via a top lateral interconnect component. At least two lightweight cache controllers, matching the number of the at least two high-bandwidth memory stacks, are disposed along the path of the top interconnect layer; the correspondence between the lightweight cache controllers and the high-bandwidth memory stacks is a one-to-one mapping; wherein the lightweight cache controller matching any one of the high-bandwidth memory stacks is used to manage the transmission of the target data on one side of that high-bandwidth memory stack; wherein the top lateral interconnect component includes any one of a top bridging die, a micro SoIC-X bridge, and an optical interconnect CPO module.

[0038] As one specific implementation method, refer to Figure 1The top interconnect layer (TIL) is physically positioned on top of two high-bandwidth memory (HBM) stacks. This interconnect spans and connects the top DRAM die layers of the two stacks via a top lateral interconnect component. It can be implemented using any advanced packaging technology such as top bridging die, micro SoIC-X bridge, or optical interconnect CPO (co-packaged optics) module to create a dedicated high-speed physical channel at the very top of the stack.

[0039] On this TIL path, a lightweight cache controller is integrated that precisely matches the number of interconnected HBM stacks. For example... Figure 1 As shown, when HBM stack A and stack B are interconnected via TIL, LCC-A1 and LCC-B1 are deployed on the interconnection path. These two controllers have a one-to-one mapping relationship with the two HBM stacks: LCC-A1 is dedicated to managing all target data transfers via the TIL path on the HBM stack A side; correspondingly, LCC-B1 is dedicated to managing data transfers on the HBM stack B side. Optionally, on this TIL path, the lightweight cache controller can be a feature-enhanced embedded DMA controller. Its design retains lightweight and high-efficiency characteristics while integrating more complex logic such as a priority arbiter, making it more suitable for high-bandwidth, large-data-volume block transfer tasks via the TIL path.

[0040] During operation, the entire process is controlled by the main control logic unit (such as...) in the bottom logic die. Figure 1 The decision is initiated by the "master controller" (marked in the diagram). When the master controller determines that a data access request (such as a large-scale data page migration) is suitable for using the high-bandwidth TIL path, it sends a start command containing complete transmission parameters to the corresponding LCC it manages (e.g., LCC-A1 on the stack A side). After LCC-A1 is triggered, it immediately performs a signaling handshake and coordination with the LCC-B1 on the opposite side through the TIL physical link, jointly establishing a point-to-point direct data channel. Subsequently, LCC-A1 is responsible for driving and managing the data transmission or reception operations on its HBM stack A side, efficiently completing cross-stack data transfer, thereby freeing the main processor from specific data transmission tasks and fully leveraging the high bandwidth advantage of the top interconnect layer.

[0041] As an optional implementation, the path selection control unit is a main control logic unit; the main control logic unit is integrated in the bottom logic die.

[0042] As one specific implementation method, refer to Figure 1 The path selection control function is implemented by a master control logic unit (also known as the master control unit), which can be integrated into the bottom logic die of each high-bandwidth memory (HBM) stack. Figure 1As shown, at the bottom of HBM stack A and stack B, the "bottom logic die" integrates the "master controller", which is the master controller logic unit.

[0043] As the local control core and global path decision center of its HBM stack, the master control logic unit operates as follows: When a computing unit within the stack or an external processor initiates a request to access data within a remote HBM stack, the request is first sent to the master control logic unit located on the local bottom logic die. The master control logic unit then analyzes the access request, dynamically executing a path selection algorithm based on multiple factors such as data block size, access latency requirements, and the availability and congestion status of current interconnect paths. The decision is to select the optimal path from the three available physical paths: the bottom interconnect layer, the middle interconnect layer, and the top interconnect layer. For example, for low-latency, small-data-volume requests, the MIL path might be preferred; for high-bandwidth, large-data-volume transmissions, the TIL path might be preferred.

[0044] Once a path is selected, the master control logic unit does not directly handle the specific data transfer operations. Instead, it issues precise transfer control commands to the lightweight cache controllers deployed on the selected path (i.e., LCC-A2 / LCC-B2 on MIL or LCC-A1 / LCC-B1 on TIL). These commands contain key parameters such as source / destination addresses and data length, thereby triggering and authorizing the corresponding LCC to execute the actual cross-stack data transfer task. After the task is completed, the LCC can report the completion status back to the master control logic unit. This control architecture, combining centralized decision-making (master control unit) with distributed execution (path LCCs), ensures intelligent and flexible scheduling of the global data flow while offloading efficient data transfer tasks to dedicated hardware execution units, thus optimizing the overall system performance and efficiency.

[0045] As an optional implementation, the path selection control unit (i.e., the main control logic unit) executes a selection strategy that is an intelligent decision-making process. Combined with... Figure 1 The three available paths shown (i.e., the bottom interconnect layer path, the middle interconnect layer path, and the top interconnect layer path) are governed by the following strategy: When the latency requirement of the target data access request is higher than a first threshold, or the requested data volume is less than a second threshold, the main control logic unit will select the middle interconnect layer path to utilize its low latency characteristics. When the data volume of the target data access request is greater than a third threshold, or the requested Quality of Service (QoS) priority is higher than a fourth threshold, the main control logic unit will select the top interconnect layer path to utilize its high bandwidth potential. For regular target data access requests that do not meet the above special conditions, the main control logic unit will select the bottom interconnect layer path as the default, stable, and reliable communication channel.

[0046] As an optional implementation, the Lightweight Cache Controller (LCC) includes an FSM control unit, a request scheduling logic unit, a register group, a local cache unit, and an interface logic unit. The input of the FSM control unit is communicatively connected to the output of the path selection control unit; the first output of the FSM control unit is communicatively connected to the control input of the request scheduling logic unit; the second output of the FSM control unit is communicatively connected to the control input of the register group; the output of the request scheduling logic unit is communicatively connected to the input of the interface logic unit; the output of the register group is communicatively connected to the control input of the local cache unit; and the output of the local cache unit is communicatively connected to the input of the interface logic unit.

[0047] Specifically, refer to Figure 2 In terms of connectivity, the FSM control unit is the core control hub of the entire LCC. Its input is communicatively connected to the output of the path selection control unit (i.e., the main control unit) located in the bottom logic die, used to receive a start command from the main control unit containing "path selection" information and complete transmission parameters (such as source / destination address, data volume). Based on this command, the FSM control unit coordinates its internal operations through its two outputs: its first output is communicatively connected to the control input of the request scheduling logic unit, used to issue transmission trigger and scheduling commands; its second output is communicatively connected to the control input of the register group, used for initialization configuration and runtime control of the register group. The output of the request scheduling logic unit is communicatively connected to the input of the interface logic unit, responsible for converting the scheduled transmission request into specific interface driver commands. The output of the register group is communicatively connected to the control input of the local cache unit, providing it with key parameters such as address mapping and transmission length. The output of the local cache unit is also communicatively connected to the input of the interface logic unit, serving as a buffer and relay station for data transmission.

[0048] During operation, the various modules within the LCC work in sequence to perform its lightweight DMA-like data transfer function. First, the FSM control unit is activated upon receiving instructions from the master control unit and enters the corresponding working state. On one hand, it writes the received transmission parameters into the register set via the "Configuration and Control" signal line; on the other hand, it initiates the request scheduling logic unit via the "Trigger and Command" signal line. Once triggered, the request scheduling logic unit determines the execution timing based on the current state and initiates an operation request to the interface logic unit at the appropriate time via the "Schedule and Trigger" signal. Simultaneously, the configured parameters in the register set are sent to the local cache unit through the "Parameter and Configuration" path, preparing it for data temporary storage, normalization, or alignment. Finally, driven by the scheduling command, the interface logic unit interconnects with the peer through a high-speed SerDes or bare wire interface. On one hand, it acquires or stores data from the local cache unit via the "Data Exchange" path; on the other hand, it performs actual data packet transmission and reception across the bridging path, thus efficiently and automatically completing a data block transfer task between HBM stacks, operating autonomously throughout without main processor intervention.

[0049] As described above, the packaging structure for multi-stack high-bandwidth memory provided in this disclosure integrates a distributed path selection control unit in the bottom logic die of each high-bandwidth memory stack and constructs a multi-dimensional communication network including bottom, middle, and top interconnect layers. This enables the local path selection control unit to intelligently select the optimal path based on data characteristics (such as latency requirements, data volume, and priority) when a stack needs to access remote data. It also coordinates with the lightweight cache controller (LCC) matching the local stack and the target stack on the optimal path to directly establish a data transmission channel between the stacks, thereby achieving direct memory access bypassing the main processor. This architecture effectively eliminates the latency bottleneck and bandwidth occupation caused by the main processor in traditional architectures, significantly improving the memory access flexibility, parallel processing efficiency, and overall system energy efficiency within the packaging structure.

[0050] Example 2 This embodiment provides a data communication method for an encapsulation structure, wherein the encapsulation structure is the multi-stack high-bandwidth memory encapsulation structure described in Embodiment 1. (Refer to...) Figure 3 The data communication method includes: S110, the path selection control unit receives a target data access request from the local computing module regarding the target data. The local computing module is integrated on the bottom logic die of the local high-bandwidth memory stack within the packaged structure.

[0051] Specifically, refer to Figure 4 Local computing module (such as Figure 4The “Compute Module A” shown in the diagram initiates a data access request for specific target data. This compute module is physically integrated on the bottom logic die of the local high-bandwidth memory stack (e.g., HBM stack A). The request is sent to the path selection control unit (i.e., the master control logic unit), which is also integrated on the same bottom logic die. This master control logic unit, as the local control core of this stack, is responsible for receiving and processing such memory access instructions.

[0052] S120. In response to the path selection control unit determining that the target data is located in a remote high-bandwidth memory stack based on the target data access request, the control unit selects a target path from at least two lateral interconnect communication paths according to a predetermined selection strategy, and sends a data request instruction for the target data to the first lightweight cache controller that matches the local high-bandwidth memory stack in the target path.

[0053] Specifically, refer to Figure 4 In response to the request, the main control logic unit first parses and determines the physical location of the target data. When it is determined that the target data is located in a remote high-bandwidth memory stack (e.g., HBM-B), the main control logic unit then makes an intelligent decision based on a selection strategy preset in the hardware or firmware. This strategy comprehensively considers factors such as the size of the requested data, access latency requirements, and the availability and load status of the current horizontal interconnect communication paths (i.e., bottom interconnect layer, middle interconnect layer, and top interconnect layer). As a specific example, for a low-latency request, the main control logic unit may preferentially choose the middle interconnect layer path. After the decision is made, the main control logic unit generates a data request instruction containing parameters such as source / target address and data length, and sends this instruction via the communication link to the first lightweight cache controller set on the selected target path, which matches the local high-bandwidth memory stack. For example, if the MIL path is selected, the instruction is sent to LCC-A2 in the path bridging structure corresponding to HBM stack A.

[0054] S130, the first lightweight cache controller sends the data request instruction to the second lightweight cache controller that matches the remote high-bandwidth memory stack in the target path.

[0055] Specifically, refer to Figure 4 Upon receiving instructions from the main control logic unit, the first lightweight cache controller (LCC-A2) activates its internal FSM control unit to coordinate the request scheduling logic and interface logic. Subsequently, LCC-A2 uses the physical medium (such as...) of its target path (i.e., the MIL path) to... Figure 4The “Bridge Die / Si Bridge” sends the encapsulated data request command to a second lightweight cache controller (i.e., LCC-B2 corresponding to HBM-B) on the same path that matches the remote high-bandwidth memory stack. This process establishes a direct, point-to-point control and data channel from the local LCC to the remote LCC.

[0056] S140. In response to receiving the data request instruction, the second lightweight cache controller obtains the target data corresponding to the data request instruction from the remote high-bandwidth memory stack.

[0057] Specifically, refer to Figure 4 After receiving a data request instruction through the interface logic unit, the second lightweight cache controller (LCC-B2) on the remote HBM stack side parses the instruction using its internal command decoding unit to obtain the accurate address information of the data to be accessed. Subsequently, the LCC-B2 initiates the actual data read operation to its attached remote high-bandwidth memory stack (HBM-B) through its managed local cache unit and address generation unit, thereby retrieving the corresponding target data from the stack's memory.

[0058] S150, the second lightweight cache controller uses the target path to feed back the target data to the first lightweight cache controller.

[0059] Reference Figure 4 After successfully acquiring the target data, the second lightweight cache controller (LCC-B2) uses the established same target path (e.g., the MIL path) to transmit the obtained target data in reverse via the bridging component (i.e., Figure 4 The returned data (X) is fed back to the first lightweight cache controller (LCC-A2). After receiving the data, LCC-A2 can temporarily store it in its local cache unit, and finally deliver it to the local computing module that initially initiated the request, thus completing a complete remote data access across the high-bandwidth memory stack. The entire process is autonomously and collaboratively executed by the LCCs at both ends, which is highly efficient and does not require continuous intervention from the main processor, fully demonstrating the advantages of a lightweight DMA controller.

[0060] As an optional implementation, the at least two horizontal interconnection communication paths include any two of the bottom interconnection layer path, the middle interconnection layer path, and the top interconnection layer path.

[0061] As an optional implementation, the specific implementation of step S120, "selecting a target path from at least two horizontal interconnection communication paths according to a predetermined selection strategy," is as follows: When the access latency required by the target data access request is higher than a first preset threshold, or the requested data amount is less than a second preset threshold, the main control logic unit will preferentially select the middle interconnect layer path. This decision stems from the fact that the MIL path, as a "short-circuit" channel embedded between DRAM die layers, has the inherent advantages of short physical distance and fast signal transmission, making it extremely suitable for the transmission of latency-sensitive small data blocks (such as cache lines). After this decision is triggered, the system will follow the process shown in the attached figure (i.e., "selecting the MIL path" in the example figure): the main control logic unit issues an instruction to the lightweight cache controller (e.g., LCC-A2) that matches the local stack on the MIL path, initiating efficient data transport via this low-latency path.

[0062] When the amount of data involved in the target data access request exceeds a third preset threshold, or the priority of the requested data transmission service quality is higher than a fourth preset threshold, the main control logic unit will preferentially select the top interconnect layer path. This decision is based on the fact that TIL paths typically provide higher in-package interconnect bandwidth, and their integrated embedded DMA controller (LCC) has a more robust priority arbitration mechanism, making them more suitable for large-scale data page migrations or high-priority batch data transmission tasks.

[0063] If the characteristics of the current data access request do not meet any of the above conditions such as high latency sensitivity, small data volume, large data volume, or high priority, the main control logic unit will default to selecting the bottom interconnect layer path. The BIL path, as a traditional interconnect method through the silicon interposer or package substrate, provides a stable and reliable default communication channel for handling general cross-stack data access without special performance requirements.

[0064] As an optional implementation, the data communication method further includes: in response to receiving the target data, the first lightweight cache controller caches the target data and / or feeds the target data back to the local computing module.

[0065] Specifically, refer to Figure 4 When the first lightweight cache controller (e.g., LCC-A2) successfully receives the requested target data from the second lightweight cache controller (LCC-B2) via the selected target path (e.g., the MIL path), its function is not limited to simple pass-through.

[0066] Upon receiving the target data, the first lightweight cache controller may perform one or both of the following operations according to a preset strategy or configuration: First, the target data is cached. Specifically, the LCC-A2 can use its internally integrated local cache unit as temporary storage to temporarily store the acquired target data ("Data X" in the figure). This caching behavior is designed to serve potential, near-future accesses. If the local computing module or other requesters request the same data again within a short period of time, the LCC-A2 can directly provide it from its local cache, thereby completely avoiding the subsequent lengthy cross-stack access latency and significantly improving data reuse efficiency and system response speed.

[0067] Second, the target data is fed back to the local computing module. Specifically, LCC-A2 can perform a proactive, reliable data feedback action to deliver the data to the original requester (i.e., "Compute Module A" in the diagram). This "feedback" operation can be designed as a confirmation mechanism or data pre-setting step to ensure that the computing module can obtain the required data in a timely and reliable manner, or to prepare data for subsequent computing tasks.

[0068] As described above, the data communication method for packaged structures provided in this disclosure constructs a multi-level horizontal interconnect path including a bottom interconnect layer (BIL), a middle interconnect layer (MIL), and a top interconnect layer (TIL) between multiple HBM stacks. It innovatively deploys a lightweight cache controller (LCC) in the bridging structure of the MIL and TIL paths, achieving intelligent dynamic optimization of path selection. This significantly reduces the latency of cross-stack data access and greatly improves effective bandwidth. The method separates global path selection decisions (handled by the main control unit in the bottom logic die) from specific data transmission execution (handled by dedicated LCCs on each path). This allows the system to automatically select the optimal communication path based on the real-time characteristics of data access requests (such as latency requirements, data size, and priority). This not only effectively avoids the bandwidth bottleneck and long latency of traditional single bottom paths, but also significantly reduces power consumption and controller resource overhead during communication due to the lightweight and dedicated design of its LCCs. This provides a high-efficiency, low-power in-package interconnect solution for applications requiring close collaboration between high-bandwidth memory and high-performance computing, AI acceleration, and other applications.

[0069] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0070] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0071] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0072] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0073] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0074] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0075] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A packaging structure for multi-stack high-bandwidth memory, characterized in that, The encapsulation structure includes a multi-layer interconnect network, at least two high-bandwidth memory stacks, and a path selection control unit that matches each of the high-bandwidth memory stacks. The multi-layer interconnect network is disposed between the at least two high-bandwidth memory stacks, and the path selection control unit is connected to the multi-layer interconnect network. Each of the high-bandwidth memory stacks includes multiple vertically stacked DRAM dies and a bottom logic die; The multi-layer interconnection network includes at least two horizontal interconnection communication paths for data communication between the at least two high-bandwidth memory stacks. The path selection control unit is configured to select a target path from the at least two lateral interconnect communication paths for target data transmission in response to a cross-stack target data access request, according to a predetermined selection strategy.

2. The packaging structure according to claim 1, characterized in that, The at least two horizontal interconnect communication paths include at least two selected from the following groups: bottom interconnect layer, middle interconnect layer, and top interconnect layer.

3. The packaging structure according to claim 2, characterized in that, The bottom interconnect layer is disposed at the bottom of the at least two high-bandwidth memory stacks, and interconnection between the bottom logic dies of the at least two high-bandwidth memory stacks is realized through a bottom horizontal interconnect component; The bottom horizontal interconnect component includes either a silicon interposer or an embedded silicon bridge.

4. The packaging structure according to claim 2, characterized in that, The central interconnect layer is embedded between the central DRAM die layers of the at least two high-bandwidth memory stacks and interconnected through laterally arranged bridging components; At least two lightweight cache controllers, matching the number of the at least two high-bandwidth memory stacks, are provided on the middle interconnect layer path; The correspondence between the lightweight cache controller in the at least two lightweight cache controllers and the high-bandwidth memory stack in the at least two high-bandwidth memory stacks is a one-to-one mapping; The lightweight cache controller, which is matched with any of the high-bandwidth memory stacks, is used to manage the transmission of the target data on one side of the high-bandwidth memory stack. The bridging component includes any one of the following: bridging die with microbumps, optical interconnect module, and micro SoIC bridge.

5. The packaging structure according to claim 2, characterized in that, The top interconnect layer is disposed on top of the at least two high-bandwidth memory stacks and interconnected through a top horizontal interconnect component; At least two lightweight cache controllers are provided on the top interconnect layer path, matching the number of the at least two high-bandwidth memory stacks; The correspondence between the lightweight cache controller in the at least two lightweight cache controllers and the high-bandwidth memory stack in the at least two high-bandwidth memory stacks is a one-to-one mapping; The lightweight cache controller, which is matched with any of the high-bandwidth memory stacks, is used to manage the transmission of the target data on one side of the high-bandwidth memory stack. The top lateral interconnect component includes any one of a top bridging die, a micro SoIC-X bridge, or an optical interconnect module.

6. The packaging structure according to claim 1, characterized in that, The path selection control unit is the main control logic unit; The main control logic unit is integrated in the bottom logic die.

7. The packaging structure according to claim 2, characterized in that, The selection strategy includes: If the latency requirement of the target data access request is higher than the first threshold, or the amount of data requested is less than the second threshold, then the MIL path of the middle interconnect layer is selected. If the data volume of the target data access request is greater than the third threshold, or the service quality priority of the requested service is higher than the fourth threshold, then the top interconnection layer TIL path is selected. Otherwise, select the bottom interconnect layer (BIL) path.

8. The packaging structure according to claim 4 or 5, characterized in that, The lightweight cache controller includes an FSM control unit, a request scheduling logic unit, a register group, a local cache unit, and an interface logic unit; The input terminal of the FSM control unit is communicatively connected to the output terminal of the path selection control unit; The first output terminal of the FSM control unit is communicatively connected to the control input terminal of the request scheduling logic unit; The second output terminal of the FSM control unit is communicatively connected to the control input terminal of the register group; The output of the request scheduling logic unit is communicatively connected to the input of the interface logic unit; The output of the register group is communicatively connected to the control input of the local cache unit; The output of the local cache unit is communicatively connected to the input of the interface logic unit.

9. A data communication method for an encapsulated structure, characterized in that, The encapsulation structure is the encapsulation structure for multi-stack high-bandwidth memory as described in any one of claims 1 to 8, and the data communication method includes: The path selection control unit receives a target data access request for the target data initiated by the local computing module; wherein, the local computing module is integrated on the bottom logic die of the local high-bandwidth memory stack in the package structure; In response to the path selection control unit determining that the target data is located in a remote high-bandwidth memory stack based on the target data access request, the control unit selects a target path from at least two lateral interconnect communication paths according to a predetermined selection strategy, and sends a data request instruction for the target data to the first lightweight cache controller that matches the local high-bandwidth memory stack in the target path. The first lightweight cache controller sends the data request instruction to the second lightweight cache controller that matches the remote high-bandwidth memory stack in the target path; In response to receiving the data request instruction, the second lightweight cache controller obtains the target data corresponding to the data request instruction from the remote high-bandwidth memory stack; The second lightweight cache controller uses the target path to feed the target data back to the first lightweight cache controller.

10. The data communication method according to claim 9, characterized in that, The at least two horizontal interconnection communication paths include any two of the bottom interconnection layer path, the middle interconnection layer path, and the top interconnection layer path; The step of selecting a target path from at least two lateral interconnection communication paths according to a predetermined selection strategy includes: If the latency requirement of the target data access request is higher than the first threshold, or the amount of data requested is less than the second threshold, then the middle interconnection layer path is selected. If the data volume of the target data access request is greater than the third threshold, or the service quality priority of the requested service is higher than the fourth threshold, then the top interconnection layer path is selected. Otherwise, select the bottom interconnect layer path.

11. The data communication method according to claim 9, characterized in that, The data communication method further includes: In response to receiving the target data, the first lightweight cache controller caches the target data and / or feeds the target data back to the local computing module.

Images