3D stacked chip heterogeneous on-chip network and data transmission method thereof

Abstract

The application provides a 3D stacked chip heterogeneous network-on-chip and a data transmission method thereof. The 3D stacked chip comprises a plurality of silicon chips stacked vertically, and a heterogeneous network-on-chip is arranged in each silicon chip. The heterogeneous network-on-chip comprises a main data network and a convergence network. The main data network comprises a plurality of main data routers, and each main data router is connected to at least one processing unit. The convergence network comprises a plurality of convergence routers, and each convergence router is connected to a subnetwork composed of a plurality of main data routers. The convergence routers are connected to each other and connected to the convergence routers of adjacent silicon chips through a cross-silicon interconnection structure. The application builds a hierarchical collaborative architecture of the main data network and the convergence network in the 3D stacked chip, and cooperates with an independent control network, so as to realize effective separation of data flow and control flow, optimization of cross-silicon communication paths, and comprehensive improvement of system throughput, real-time performance and reliability.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and more specifically, to a heterogeneous on-chip network for 3D stacked chips and its data transmission method. Background Technology With the continuous advancement of integrated circuit technology and the explosive growth in computing power demands from applications such as artificial intelligence and high-performance computing, 3D stacked chip technology has become a key approach to breaking through Moore's Law bottlenecks and achieving higher integration and performance. Vertically stacking multiple silicon dies using through-silicon vias (TSVs) can significantly improve chip bandwidth, reduce power consumption, and support heterogeneous integration. In 3D stacked chips, the Network on Chip (NoC) serves as the core interconnect architecture connecting multiple processing elements (PEs), and its design directly determines the system's data transmission efficiency, latency, and overall performance.

[0002] In existing technologies, 3D stacked chips typically deploy the same or similar two-dimensional NoCs (such as Mesh topology or a hybrid Mesh and Crossbar topology) within each silicon layer. Inter-layer communication is simply extended via TSVs, symmetrically connecting router nodes located at different silicon wafers. Routing algorithms and flow control mechanisms remain largely consistent within and between layers, essentially representing a simple replication and extension of the two-dimensional NoC in the three-dimensional die dimension. This approach is relatively mature in two-dimensional chips, but when directly applied to 3D stacking scenarios, it lacks specific optimizations for vertical communication characteristics.

[0003] However, the aforementioned existing technical solutions have the following main shortcomings in practical applications: First, the communication modes are mismatched. In scenarios such as Large Language Model (LLM) inference, there are simultaneously high-bandwidth data flows (such as weights and activation values) and low-latency control flows (such as key-value pairs). The mixed transmission of cache cross-PE access and synchronization instructions can easily cause head-of-line congestion, leading to a sharp drop in the overall throughput of the NoC. Secondly, cross-layer communication efficiency is low. The latency and bandwidth characteristics of TSV links are significantly different from those of intra-layer metal interconnects. However, in existing solutions, cross-layer traffic and intra-layer traffic share the same path, which can easily form traffic hotspots, causing communication bottlenecks in processing units and reducing the overall token generation rate. Thirdly, thermal management capabilities are insufficient. The vertical thermal stacking problem of 3D stacked chips is more serious than that of traditional two-dimensional chips. The control of cross-die traffic in existing homogeneous NoCs mainly relies on the backpressure signal of the TSV physical layer. The protocol layer information has been lost, and it is impossible to achieve active throttling and global control of hotspots at the network protocol level. Moreover, when local overheating occurs, due to the lack of a control mechanism involving the protocol layer, it is difficult to effectively constrain cross-silicon data transmission in a timely manner. It is impossible to finely control data transmission behavior based on temperature status. If overheating occurs, data must be temporarily stored locally and sent after the temperature drops.

[0004] The aforementioned problems have become the main technical obstacles restricting the further development of 3D stacked chips in the field of high-performance computing. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a 3D stacked chip heterogeneous on-chip network and its data transmission method. By constructing a hierarchical network architecture consisting of a main data network and a convergence network within each silicon chip, and structurally defining the communication path in the cross-silicon chip direction, the separation and optimized scheduling of different types of communication traffic can be achieved.

[0006] Specifically, this invention sets up a main data network to carry high-bandwidth data transmission within the silicon wafer, and uses an aggregation network to aggregate multiple main data routers at the subnet level. This ensures that cross-subnet and cross-wafer communication is uniformly completed through the aggregation network, thereby avoiding interference from cross-wafer traffic to the main data network, effectively reducing head-of-line congestion problems, and improving overall throughput. Simultaneously, the cross-wafer interconnection structure between aggregation routers enables centralized management of cross-layer communication paths, reducing the number of hops in cross-layer communication, alleviating TSV link hotspot issues, and improving cross-wafer data transmission efficiency.

[0007] Furthermore, this invention introduces a control network independent of the data network for transmitting control signals, configuration commands, and synchronization information, achieving physical isolation between the control flow and the data flow. This ensures the real-time performance of low-latency control commands and prevents control signals from being blocked during large-scale data transmission. In addition, by introducing a temperature-based flow control mechanism at the protocol layer of the convergence network, and combining it with the temperature status information and flow control information transmitted by the control network, the system can actively adjust the cross-wafer flow at the protocol layer, effectively mitigating the thermal buildup problem in 3D stacked chips and improving system operational stability.

[0008] Therefore, this invention overcomes the problems of communication mode mismatch, low cross-layer communication efficiency and insufficient thermal management capability in the prior art by coordinating the design of heterogeneous hierarchical network architecture, cross-silicon chip communication path constraints and control and data separation mechanism, and achieves comprehensive optimization of bandwidth utilization, transmission delay and system reliability.

[0009] In a first aspect, the present invention provides a heterogeneous on-chip network for a 3D stacked chip, characterized in that the 3D stacked chip comprises multiple vertically stacked silicon wafers, the heterogeneous on-chip network is disposed within each of the silicon wafers, and the heterogeneous on-chip network comprises a main data network and a convergence network; wherein The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

[0010] Secondly, the present invention also provides a data transmission method for a heterogeneous on-chip network of a 3D stacked chip, characterized in that it is applied to a 3D stacked chip comprising multiple vertically stacked silicon wafers, wherein the heterogeneous on-chip network is disposed within each of the silicon wafers, and the heterogeneous on-chip network includes a main data network and a convergence network; wherein The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected with each other and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure. The method includes: Data is transmitted between processing units connected to the main data router through the main data router in the main data network; Data is transmitted between subnets consisting of multiple primary data routers connected to the aggregation router in the aggregation network, and between the interconnected aggregation routers, as well as with aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

[0011] Compared with the prior art, the present invention has at least the following beneficial effects: Firstly, regarding communication performance, this invention constructs a hierarchical structure of a main data network and an aggregation network within each silicon chip. This allows high-bandwidth data streams to be primarily transmitted in the main data network, while cross-subnet and cross-silicon chip communication is uniformly handled through the aggregation network. This effectively separates different types of data streams, avoids cross-silicon chip traffic from occupying intra-layer communication paths, significantly reduces head-of-line congestion, improves overall network throughput, and reduces data transmission latency. Simultaneously, the main data routers in the main data network are stacked to form main router clusters. Each main router cluster includes at least two layers of main data routers arranged vertically, thus forming multi-layer parallel data transmission channels within the same subnet. This significantly increases data transmission bandwidth and enhances the parallel carrying capacity for bursty data streams. The aggregation network provides additional buffer resources and path selection capabilities. By redirecting and distributing data streams in higher-layer networks, it reduces local link congestion, thereby improving the system's stable throughput under high-load scenarios.

[0012] Secondly, regarding communication path optimization, this invention introduces aggregation routers at the subnet level and constructs an aggregation network as a higher-level interconnection structure. This enables cross-subnet communication to be completed with fewer hops, while cross-silicon communication is centrally forwarded through the aggregation routers, reducing path redundancy issues associated with traditional hop-by-hop mesh paths. Furthermore, a multi-path coordination mechanism is formed between the main data network and the aggregation network, allowing data packets to choose whether to undergo accelerated transmission via the aggregation network, provided that quality of service requirements are met. This improves path flexibility and overall scheduling efficiency.

[0013] Furthermore, regarding service quality and real-time performance assurance, this invention establishes an independent control network, separating control signals, configuration commands, and synchronization information from the data flow on the physical link and protocol path. This prevents control information from being blocked during high-load data transmission, thereby ensuring the real-time performance of low-latency operations such as Barrier, synchronization control, and flow control feedback. Simultaneously, the protocol-level credit synchronization and backpressure mechanism implemented through the control network can quickly transmit feedback signals upstream when network congestion trends occur, thereby reducing the probability of congestion and significantly shortening congestion recovery time.

[0014] Furthermore, in terms of system stability and thermal management, this invention introduces a temperature control and flow control mechanism at the protocol layer of the convergence network. By controlling the transmission of temperature status information and flow control signals through the control network, the system can dynamically adjust the cross-silicon wafer data transmission rate according to the silicon wafer temperature, thereby achieving active regulation of cross-die flow. This effectively suppresses the thermal accumulation problem in 3D stacked chips, reduces the risk of thermal throttling and thermal failure, and improves the reliability of system operation.

[0015] Finally, in terms of resource utilization efficiency, this invention aggregates subnets and centrally configures cross-silicon interconnect structures, enabling unified management of cross-silicon communication resources at the aggregation layer. This avoids setting up cross-silicon interfaces at all main data routers, thereby significantly reducing the number of cross-silicon interconnect resources (such as TSVs), reducing chip area overhead, and improving overall resource utilization efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the main data network and aggregation network structure of the 3D stacked chip provided in an embodiment of the present invention; Figure 2 This is provided by the embodiments of the present invention. Figure 1 A schematic diagram of the control network logic corresponding to the 3D stacked chip; Figure 3 This is a flowchart of a data transmission method for heterogeneous on-chip networks of 3D stacked chips provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview As mentioned above, this invention provides a heterogeneous on-chip network for 3D stacked chips and its data transmission method. By constructing a hierarchical collaborative architecture of a main data network and a convergence network in the 3D stacked chip, and in conjunction with an independent control network, it achieves effective separation of data flow and control flow, optimization of cross-silicon communication paths, and comprehensive improvement of system throughput, real-time performance, and reliability.

[0020] Exemplary System Figure 1 This is a schematic diagram of the main data network and aggregation network structure of the 3D stacked chip provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the 3D stacked chip includes multiple vertically stacked silicon dies, and the heterogeneous on-chip network is disposed within each of the silicon dies. The heterogeneous on-chip network includes a main data network, a convergence network, and a control network. It should be noted that, for clear illustration of the data path, Figure 1 The control network is not included, nor is the processing element (PE) connected to each main router cluster shown. The three types of networks are structurally independent but functionally collaborative, jointly supporting the data transmission and control requirements within the chip. They form a heterogeneous NoC hierarchical system within the same silicon chip, achieving joint optimization of data transmission bandwidth, latency, and control signal real-time performance through hierarchical support of different types of communication tasks.

[0021] For example, the primary data network corresponds to a 2D Mesh primary data network (NoC) and is used to carry large-scale data streams; the aggregation network corresponds to a 2D Mesh aggregation network (NoC) and is used to provide fast paths across subnets and undertake cross-silicon communication; the control network corresponds to a tree-structured control network (NoC) and is used to carry control, configuration, and synchronization signals.

[0022] First, the structure of the main data network will be explained: The main data network includes multiple main data routers, each connected to at least one processing unit, for carrying large-scale data stream transmission including activation values ​​and partial weight blocks, and features high throughput, regular topology and easy scalability.

[0023] The multiple master data routers in the master data network constitute multiple master router clusters. Each master router cluster contains a single master data router, or contains at least two master data routers stacked vertically, providing high-bandwidth point-to-point communication. Each master router cluster is connected to a convergence router. The processing unit is connected to the master router cluster.

[0024] The primary data routers in the primary data network employ either a credit-based flow control mechanism or a handshake-based flow control mechanism. Credit-based flow control maintains a "credit value" between adjacent routers, representing the available buffer resources at the receiving end. The sending end is only allowed to send data when it holds sufficient credit value. When the receiving end releases its buffer resources, it returns the corresponding credit information to the sending end, thus achieving precise control over the data injection rate. This mechanism enables pipelined transmission without adding extra handshake latency and is suitable for high-bandwidth, long-distance data paths. Handshake-based flow control, on the other hand, uses request / acknowledge signals to control data transmission. The sending end waits for the receiving end to return a permission signal before sending data, and the receiving end sends an acknowledgment signal after confirming receipt, ensuring that each data transmission occurs only when the receiving end has the processing capability. This mechanism is simple to implement, offers fine-grained control, and is suitable for communication scenarios with low bandwidth or high reliability requirements.

[0025] By supporting at least one of the two flow control mechanisms mentioned above in the main data network, flexible selection can be made according to different application scenarios and link characteristics: a credit-based flow control mechanism is used in high-throughput data transmission paths to improve link utilization and reduce transmission latency; a handshake-based flow control mechanism is used in resource-constrained or timing-critical paths to enhance the determinism and reliability of transmission.

[0026] Therefore, by introducing a configurable flow control mechanism, this invention can not only effectively avoid data loss caused by buffer overflow or link congestion, but also improve the stability and throughput performance of the main data network under high load conditions, thereby enhancing the data transmission efficiency and operational reliability of the entire 3D stacked chip heterogeneous on-chip network system.

[0027] For example, such as Figure 1 As shown, this 3D stacked chip comprises four vertically stacked silicon dies. Each die layer houses a 4x4 main data network, meaning each layer contains 16 main data router clusters. Each main data router cluster consists of multiple stacked main data routers, such as... Figure 1As shown in the diagram of "Layer 1 Master Data Router" and "Layer 2 Master Data Router," each cluster contains at least two master data routers in the vertical direction, located at different vertical layers (Layer 1 and Layer 2). These master data router clusters are interconnected via master data network connections within the die. In practice, each master data router cluster connects to one or more processing units (PEs) to carry the transmission of large-scale data streams such as activation values ​​and weights. Trust-based or handshake-based flow control mechanisms can be used between master data routers to ensure the reliability of data transmission.

[0028] Based on the aforementioned main data network, a convergence network for cross-subnet and cross-silicon wafer communication is further established, with the following structure: The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure, thereby concentrating cross-wafer data transmission capabilities at the aggregation network level.

[0029] In this network, each aggregation router connects to a subnet consisting of N×M (e.g., 2×2, where N and M may be unequal) master data routers, where N and M are positive integers and N×M≥2, to achieve coarser-grained packet management compared to the master data network. All master router clusters within a subnet are connected to the same aggregation router, which in turn is directly connected to all processing units within that subnet via network interfaces. The aggregation routers form a higher-level topology and are connected to a cross-die interconnect structure to enable cross-die data transmission.

[0030] The aggregation router establishes a communication connection with the processing unit in its corresponding subnet through a network interface, which is used to provide a fast cross-subnet path that bypasses the multi-hop main data network within the same silicon chip.

[0031] Furthermore, to enable data interaction between different silicon wafers, the cross-wafer interconnect structure specifically includes the following forms: The cross-silicon interconnect structure includes through-silicon vias (TSVs), silicon interposers, or swapped dies.

[0032] Regarding network topology and routing mechanisms, the present invention can also be implemented in the following ways: The main data network and / or the aggregation network adopts a mesh topology, which includes a two-dimensional mesh topology or a torus topology.

[0033] The mesh topology refers to a network topology where router nodes are interconnected in a regular two-dimensional manner, forming a network with high path diversity and scalability. This topology can provide rich routing paths for high-bandwidth data transmission, while facilitating hardware cabling and large-scale expansion.

[0034] A two-dimensional mesh topology, where each router is directly connected to its four adjacent routers (top, bottom, left, and right), forms a regular two-dimensional array. It features simple wiring, a neat layout, and ease of physical implementation, making it ideal for achieving high-throughput data transmission in the main data network of this invention. A torus topology, building upon the two-dimensional mesh topology, connects the outermost routers of the array end-to-end, forming a closed torus structure. This further enhances network path diversity and reduces the average hop count at edge nodes, making it particularly suitable for scenarios in aggregation networks requiring fast cross-subnet transmission.

[0035] The primary data network and / or the aggregation network may also adopt a tree topology, including a fat-tree topology. A fat-tree topology is a hierarchical tree topology characterized by increased link bandwidth and port count as the network moves upwards, enabling non-blocking communication. In an aggregation network, it provides ample aggregation bandwidth for large amounts of cross-subnet data streams while supporting efficient multicast and broadcast operations.

[0036] It should be noted that this invention is not limited to the specific topologies listed above. Any topology that is essentially mesh-like (i.e., regularly interconnected nodes with high connectivity and scalability) falls within the scope of this invention. Those skilled in the art can flexibly select one or more combinations of topologies based on actual area, power consumption, and performance requirements to further optimize the overall performance of the heterogeneous on-chip network of this invention.

[0037] Through the design of the above topology, the main data network and aggregation network of the present invention maintain a high degree of regularity and scalability, and can also work well with the hierarchical distribution topology of the control network to achieve joint optimization of data transmission bandwidth, latency and control signals.

[0038] The main data network and / or the aggregation network employ a static routing algorithm or a congestion-aware adaptive routing algorithm.

[0039] The routing algorithm in this invention is used to determine the transmission path of data packets in the main data network or aggregation network. Static routing algorithms pre-determine a fixed transmission path for each data packet based on the network topology. They are simple to implement, have low overhead, and low routing decision latency, making them suitable for scenarios with relatively stable network load. Congestion-aware adaptive routing algorithms, on the other hand, can monitor the congestion status of each link and node in the network in real time and dynamically adjust the transmission path of data packets according to the current congestion situation, thereby effectively avoiding congestion hotspots and improving overall network throughput and transmission efficiency.

[0040] In a preferred embodiment of the present invention, the main data network and the aggregation network can flexibly select one of the two routing algorithms or a combination thereof according to actual application requirements. For example, when the network load is relatively balanced, the static routing algorithm can be used first to reduce the routing calculation overhead; while in scenarios with large bursts of traffic, such as LLM inference, the congestion-aware adaptive routing algorithm can be switched to further leverage the cross-subnet acceleration advantage of the aggregation network through dynamic path selection.

[0041] It should be noted that this invention is not limited to the two specific routing algorithms mentioned above. Any static or adaptive routing strategy capable of packet path selection falls within the scope of this invention. Those skilled in the art can select or combine the two algorithms based on chip area, power consumption, and real-time requirements to further optimize the performance of the heterogeneous on-chip network of this invention.

[0042] Through the design of the above routing algorithm, the main data network and aggregation network of this invention can achieve flexible scheduling of data transmission paths while maintaining regular topology, thus forming a good match with the structural characteristics of the three-layer heterogeneous NoC and effectively improving the overall data transmission bandwidth and latency performance.

[0043] Regarding data transmission path control, the present invention further specifies that once a data packet enters the aggregation network from the main data network, its transmission path is restricted to only the aggregation network and the cross-silicon interconnect structure, and it will not return to the main data network, so as to avoid the risk of deadlock caused by circular dependencies between different network levels.

[0044] In this invention, a strict one-way transmission constraint is followed between the main data network and the aggregation network: data packets can enter the aggregation network from the main data network (for cross-subnet acceleration or cross-die transmission), but once a data packet is forwarded from the main data network to the aggregation network, it is prohibited from being forwarded back to any main data network (including other subnets within the same silicon wafer, main data networks within different silicon wafers, etc.). After receiving a data packet from the main data network, the aggregation router is only allowed to forward it directly to the destination processing unit (PE) through the network interface, and must not resend the data packet to the main data network.

[0045] The above constraints are set to avoid forming a closed-loop communication path between the main data network and the aggregation network, thereby preventing deadlock. Deadlock refers to a state in which a group of data packets in a network is permanently blocked and unable to continue transmission because they are waiting for each other to release their buffer resources. Specifically, if both sending data packets from the main data network to the aggregation network (allowed in this scheme) and sending data packets from the aggregation network to the main data network (prohibited in this scheme) are allowed, a bidirectional path will be formed between the two networks. When the input buffers of each node are full, the following circular waiting situation may occur: Node A in the main data network waits for Node B in the aggregation network to release its buffer, Node B waits for Node C in the main data network to release its buffer, and Node C waits for Node A to release its buffer. Since each node is waiting for the next node, and no node can continue to forward data, the entire network is partially and permanently blocked, which is deadlock. Once deadlock occurs, it can usually only be resolved through a global reset or external intervention, severely affecting the normal operation of the chip.

[0046] The unidirectional transmission constraint of this invention cuts off the data path from the aggregation network to the main data network, thereby limiting the data flow direction to a unidirectional directed acyclic path of "main data network → aggregation network → PE". This completely eliminates the possibility of forming a closed loop waiting at the architectural level, thus fundamentally avoiding deadlock and ensuring the determinism of network communication.

[0047] Specifically, when the main data router receives a data packet from the processing unit, it first parses the destination address information in the packet header to determine whether the data packet needs to be transmitted across subnets or silicon wafers. If it is determined that the data packet needs to be transmitted through the aggregation network, the main data router forwards the data packet to the corresponding aggregation router and sets a "entered aggregation layer" flag in the packet header.

[0048] Upon receiving the data packet, the aggregation router queries its internal forwarding state machine. This state machine, based on a flag, forces all subsequent forwarding decisions to be confined within the aggregation network and across silicon interconnect structures (such as TSVs, interposers, or switching intermediate chips), prohibiting any routing decisions from pointing to the entry port of the main data network. This unidirectional path restriction is checked by the hardware state machine every clock cycle, ensuring the determinism and real-time nature of path control.

[0049] Furthermore, the aforementioned path restriction mechanism works closely in conjunction with the control network. When a data packet enters the aggregation network, the control network simultaneously sends an "aggregation layer occupancy" credit signal to the relevant master data router, notifying the upstream master data network that it will no longer reserve a backoff path for the data packet, thereby completely severing any potential circular dependencies at the protocol layer.

[0050] In a preferred embodiment of the invention, this restriction remains strictly effective even in the event of momentary network congestion or sudden traffic surges: the aggregation router does not forward data packets back to the main data network, but instead forwards them directly to the destination PE (within the same silicon wafer) via the network interface or sends them to the aggregation router on the target silicon wafer via a cross-wafer interconnect structure, whereby the destination aggregation router finally delivers them to the destination PE. This design fundamentally eliminates the deadlock risk caused by multi-layer network loop waiting in traditional homogeneous NoCs.

[0051] Through the aforementioned path control mechanism, this invention effectively improves the operational reliability of heterogeneous on-chip networks of 3D stacked chips while ensuring high-bandwidth local transmission. It is particularly suitable for high-performance computing scenarios such as LLM inference, which involve a large amount of cross-subnet and cross-die data interaction.

[0052] The primary data router is configured to: when the destination address of the data packet to be transmitted is within the same silicon wafer but across subnets, determine whether to forward the data packet to the aggregation router for accelerated transmission based on the quality of service requirements, data path hop count and / or the congestion status of the aggregation network, thereby dynamically balancing performance and resource consumption.

[0053] Specifically, after receiving a data packet from the processing unit, the main data router first parses the destination address field in the packet header to determine whether the address belongs to the same silicon wafer but spans different subnets. If it is determined to be a cross-subnet transmission, the path decision module is immediately activated.

[0054] The path decision module comprehensively evaluates the following three key parameters: (1) the Quality of Service (QoS) requirements of the data packets, including priority, maximum allowable delay, etc.; (2) the estimated number of hops from the source master router to the destination master router in the current master data network; and (3) the congestion status of the aggregation network synchronized in real time by the control network (reflected by protocol-level credit value or queue occupancy rate). The above three parameters are input into a preset decision threshold table or a simple weighted comparison logic to jointly form the final forwarding decision.

[0055] For example, when the QoS requirement is high priority and the number of hops on the primary data path is greater than a preset threshold (e.g., ≥4 hops), and the current credit value of the aggregation network is higher than a set threshold (i.e., the congestion level is low), the primary data router will directly forward the data packet to the corresponding aggregation router, and the aggregation network will provide short-path acceleration. Conversely, if the number of hops on the primary data path is low or the aggregation network is already in a high-congestion state, the data packet will continue to be transmitted in the primary data network in a regular multi-hop manner, thereby achieving a dynamic balance between performance improvement and resource consumption.

[0056] In a preferred embodiment of the present invention, the above decision-making process is completed by a hardware comparator inside the master data router within a single clock cycle, avoiding the introduction of additional delays; at the same time, the control network periodically broadcasts the global congestion status information of the aggregation network to all master data routers, ensuring that the decisions of each master data router are based on the latest network status.

[0057] Through the aforementioned intelligent path selection mechanism, this invention can flexibly switch transmission paths under different load conditions, which not only makes full use of the advantages of fast cross-subnet paths in the aggregation network, but also avoids unnecessary resource occupation, significantly improving the overall throughput and transmission efficiency of the entire heterogeneous on-chip network in mixed traffic scenarios such as LLM inference.

[0058] In handling cross-die data communication, the present invention, in its preferred embodiment, strictly limits cross-die data communication to only being allowed through the aggregation network, and the main data network does not directly participate in cross-die communication. This limitation effectively isolates cross-die traffic, preventing it from polluting the high-bandwidth local path of the main data network, thereby achieving joint optimization of bandwidth, latency, and thermal management.

[0059] However, to adapt to the resource availability of cross-wafer interconnect structures under different process conditions, this invention also provides another implementation. When cross-wafer interconnect structure resources are sufficient, this invention allows cross-die connections to be directly implemented on each master router cluster. In this case, the aggregation router no longer undertakes cross-die communication functions, but is only responsible for accelerating data transmission across subnets within the same wafer. That is, the master data router can directly achieve cross-die data transmission through the cross-wafer interconnect structure allocated to its master router cluster, while the aggregation network focuses on providing short-path acceleration between subnets.

[0060] Through the design of the above different implementation methods, the present invention not only ensures the creativity of the core technical solution, but also provides a flexible choice space for actual chip design. Those skilled in the art can reasonably select any of the above methods or combinations thereof based on the area cost, bandwidth margin and heat dissipation constraints of the cross-silicon interconnect structure in the specific process.

[0061] For example, Figure 1 This illustrates a 2×2 subnetting scheme for the aggregation network. Specifically, each 2×2 adjacent clusters of primary data routers form a subnet, and all primary data router clusters within the same subnet are connected to the same aggregation router. Figure 1In the 4×4 master data network shown, each silicon wafer layer forms four subnets (top left, top right, bottom left, and bottom right), corresponding to four aggregation routers. Each aggregation router is connected to the master data router cluster in its corresponding subnet via a master data network-aggregation network connection cable within the die. Simultaneously, the aggregation router is also directly connected to all processing units (PEs) within that subnet via network interfaces, enabling it to receive data packets from the master data network and directly send data packets to their destination PEs.

[0062] like Figure 1 As shown, the aggregation routers within the same silicon die are interconnected via aggregation network cables, forming a higher-level 2D Mesh topology. This higher-level mesh network provides a fast data transmission path across subnets: when a data packet within one subnet needs to be transmitted to another, the packet can first be sent from the main data network to the aggregation router of the current subnet, and then quickly forwarded to the aggregation router of the target subnet via the aggregation network. This avoids multiple hops in the main data network and reduces transmission latency.

[0063] Figure 1 The vertical interconnection across dies is also illustrated. Aggregate routers between different dies are vertically interconnected via a cross-die interconnect structure. Specifically, the aggregation routers in Layer 1 are connected to their corresponding aggregation routers in Layer 2 via a cross-die interconnect structure, and the same applies to Layer 2 and Layer 3, and Layer 3 and Layer 4. Aggregate routers in each vertical column are cascaded via the cross-die interconnect structure, forming a cross-die aggregation network topology. It is important to note that there are no cross-die connections between the main data routers. Figure 1 The main data router clusters only have horizontal intra-die connections, with no vertical cross-die interconnect structures. In a preferred embodiment of the invention, cross-die data transmission is only permitted through the aggregation network; the main data network does not participate in cross-die communication.

[0064] based on Figure 1 The data packet transmission path is as follows: In actual data transmission, when a processing unit (PE) initiates a read data request or a write data request, the request is first sent to the cluster of primary data routers it is connected to. Since each cluster of primary data routers contains multiple vertically stacked primary data routers (as shown in Layer 1 and Layer 2 in the figure), the sending end can select the less congested layer for transmission based on the current congestion status of each layer of primary data routers, in order to balance the load and reduce latency.

[0065] Upon receiving the request, the master data router then determines whether the target address spans a silicon wafer and a subnet: When data packets are transmitted within the same subnet, they are transmitted directly between the main data router clusters within the subnet via the main data network, without needing to pass through the aggregation router. When data packets need to cross subnets but remain within the same silicon wafer, the main data router clusters can choose whether to send the data packets to the aggregation network for accelerated transmission based on quality of service requirements, path hop count, and / or aggregation network congestion status. When data packets need to be transmitted across dies, the source main data router cluster sends the data packets to the aggregation router within the same subnet. This aggregation router then vertically transmits the data packets to the corresponding aggregation router on the target silicon wafer via the cross-die interconnect structure, and the aggregation router then forwards them to the target processing unit.

[0066] The aforementioned decision to accelerate transmission across subnets occurs each time a subnet is crossed; however, for data packets that have already been forwarded to the aggregation network, their transmission path is restricted to only the aggregation network and the cross-silicon interconnect structure, and they are directly forwarded to the destination PE through the network interface without returning to the main data network, otherwise a deadlock will occur.

[0067] In summary, Figure 1 The structure shown realizes the division of labor and cooperation between the main data network and the aggregation network: the main data network focuses on high-throughput data transmission within the layer, while the aggregation network provides intra-layer cross-subnet acceleration and inter-layer cross-die communication. The two types of networks are independent in topology but collaborative in function. Figure 2 The corresponding control layer network structure will be shown further.

[0068] In addition to the data transmission network, this invention also includes an independent control network, the structure of which and its functions are as follows: The control network is independent of the main data network and aggregation network in terms of physical links and protocol paths, and does not carry data transmission. The control network is used to transmit control signals, configuration commands, and synchronization signals, including protocol-level flow control backpressure and recovery, Barrier / Sync commands, etc. The main data network, aggregation network, and control network share load status information and implement collaborative scheduling based on the load status to achieve overall optimization in bandwidth utilization, transmission latency, and control real-time performance. Specifically, the control signals may include routing configuration parameters, flow control threshold parameters, congestion status indication information, and temperature control-related control information; the configuration commands are used to dynamically adjust routing strategies, buffer allocation strategies, and priority strategies in the main data network and aggregation network; the synchronization signals are used to achieve task-level or periodic-level synchronization among multiple processing units, thereby ensuring the consistent execution of parallel computing tasks. By completely isolating the above-mentioned control information from the data flow at the physical and protocol levels, the blocking of control information in high-bandwidth data transmission scenarios can be avoided, improving the determinism and real-time performance of the overall system response.

[0069] The control network adopts a hierarchical distribution topology, which includes tree topology, tree-like topology, or ring broadcast topology. The tree structure is naturally suitable for multicast / broadcast and has a short hop count, so as to ensure the transmission of control information with high real-time requirements.

[0070] Furthermore, the connection relationships and control functions of the control network in the overall system are as follows: The control network includes a root node, intermediate nodes, and terminal nodes. The terminal nodes of the control network are connected to the processing unit and the corresponding master data router. The intermediate nodes are connected to the aggregation router. The root node has cross-silicon chip communication capability and realizes cross-silicon chip communication through the cross-silicon chip interconnect structure. This enables the configuration instructions and synchronization signals from the global control unit to be consistently distributed among multiple silicon chips, thereby realizing global coordinated control of the multi-die system.

[0071] The control network is also configured to synchronize protocol-level credit information and, upon detecting network congestion, request upstream producers to send backpressure signals to participate in traffic scheduling and operational status control of the main data network and aggregation network. The protocol-level credit information refers to status information at the network transport protocol layer used to characterize the data receiving capability of downstream nodes, such as the currently available buffer space, the number of data packets that can be received, or available channel resources of a target router or link. Through this credit information, upstream nodes can determine whether they can continue sending data, thereby avoiding data loss or congestion due to insufficient downstream resources. The synchronized protocol-level credit information refers to the rapid propagation and consistent updating of the aforementioned credit status among different network nodes through the control network. This enables each node to make sending decisions based on globally or regionally consistent resource status, rather than relying solely on local link feedback, thereby improving the timeliness and accuracy of flow control response.

[0072] Specifically, the control network can collect information such as the buffer occupancy status of each router, link utilization, and packet queuing status in real time, and quickly synchronize them among network nodes through a protocol-level credit update mechanism. When a congestion trend is detected in a link or node, the control network can generate a backpressure signal and quickly transmit it back to the upstream data source to limit the data injection rate, thereby preventing congestion from spreading in the network and shortening the congestion recovery time. The backpressure signal is a control signal generated by a downstream node or the control network based on the congestion status, used to notify the upstream data producer to reduce the sending rate or suspend data injection. This backpressure signal can be quickly transmitted to relevant nodes through the control network via broadcast or directed transmission, allowing upstream nodes to dynamically adjust their sending behavior upon receiving the signal, such as reducing the packet injection rate, delaying transmission, or adjusting priorities, thereby preventing congestion accumulation or even deadlock in intermediate network paths due to continuous data injection. Through the synergistic effect of protocol-level credit information synchronization and the backpressure signal, a closed-loop flow control mechanism can be formed at the network layer, achieving fine-grained congestion control and stable operation of the main data network and aggregation network.

[0073] The convergence ratio of the root node of the control network is configured to be adjusted according to the traffic load. Specifically, the convergence ratio is used to characterize the proportion of connections from multiple downstream nodes to the upstream node. A higher convergence ratio can be used under low load conditions to reduce hardware resource consumption, while a lower convergence ratio can be used to increase control path bandwidth under high load or high concurrency control requirements, thereby improving the efficiency of control information distribution. The convergence ratio can be dynamically configured according to the real-time monitored network load to achieve a balance between resource overhead and control performance.

[0074] For example, such as Figure 2 As shown, the 3D stacked chip comprises vertically stacked multilayer silicon dies. Within each die, a control router is located, interconnected in a tree topology to form an independent control network. The final nodes of the control network connect to the processing unit (PE) and its corresponding master data router cluster; intermediate nodes connect to the aggregation router; and the root node connects to the global control unit and retains cross-die communication capabilities.

[0075] like Figure 2 As shown, the control routers are interconnected via control router connection lines, forming a tree-shaped distribution topology. This tree topology inherently supports multicast / broadcast functions, facilitating the efficient distribution of control information such as configuration commands and synchronization signals to various processing units and router nodes. Specifically, the global control unit is located at the root node of the tree network and is responsible for issuing configuration commands, Barrier / Sync commands, and protocol-level flow control backpressure and recovery signals. These signals are forwarded downstream along the tree network, eventually reaching the processing units and main data router cluster connected to the last-level nodes.

[0076] Furthermore, Figure 2 The diagram illustrates a cross-die interconnect structure. Control routers between different dies are vertically interconnected through this structure, enabling the root node of the control network to transmit signals across dies to control routers at each layer, thus achieving globally unified control and scheduling. Unlike the control network, where cross-die data communication is only permitted through the aggregation layer network, cross-die communication in the control network is independent of data transmission and is specifically used for transmitting control-related information.

[0077] exist Figure 2 In the structure shown, the control network undertakes the following functions: First, it carries the distribution of configuration instructions from the global control unit, broadcasting configuration information to all processing units and routers; Second, it broadcasts synchronization control instructions such as Barrier to ensure coordinated execution among multiple processing units; Third, it quickly synchronizes protocol-level credit information and promptly requests upstream production to send backpressure signals when congestion occurs, preventing downstream failure to receive signals and subsequent upstream congestion leading to network deadlock; Fourth, it synchronizes and aggregates congestion and temperature status information from routers and distributes it globally or locally, providing real-time data support for temperature control, flow control, and path selection; Fifth, it undertakes the transmission of some configuration information, reducing the burden on the data network.

[0078] It should be noted that the control network is independent of the main data network and aggregation network in terms of physical links and protocol paths, and does not carry large-scale data transmission. The control network is dedicated to transmitting control signals, configuration commands, and synchronization signals, thereby ensuring the transmission quality of control information with high real-time requirements and preventing it from being blocked by data streams.

[0079] In summary, Figure 2 The control network shown is Figure 1 The master data network and aggregation network shown are matched to form the three-layer heterogeneous on-chip network of this invention: Figure 1 The main data network and aggregation network are responsible for high-throughput data transmission and cross-die data communication. Figure 2 The control network in the system is responsible for global control, synchronization, and flow control. The three types of networks are physically independent but functionally collaborative, achieving joint optimization of data bandwidth, transmission latency, and control real-time performance.

[0080] In terms of thermal management and cross-wafer transmission control, this invention further achieves the following functions through the coordinated operation of a control network and a convergence network: The aggregation network is also configured to implement temperature-controlled flow control for cross-silicon wafer transmission at the protocol layer. It actively limits the cross-silicon wafer data transmission rate based on wafer temperature, thereby proactively managing the thermal issues of the 3D stacked chips at the network layer. Specifically, the aggregation router can internally include a temperature sensing module and a rate adjustment module. The temperature sensing module acquires temperature status information related to the current or adjacent silicon wafers. The rate adjustment module dynamically adjusts the injection rate, transmission window size, or scheduling priority of the cross-silicon wafer link according to a preset temperature threshold strategy. When the temperature exceeds a first threshold, the transmission rate of cross-silicon wafer data packets is reduced. When the temperature further rises above a second threshold, the cross-silicon wafer transmission of some low-priority data streams can be further restricted, thus achieving tiered flow control.

[0081] When the aggregation router is restricted from transmitting due to temperature exceeding a preset threshold, the aggregation router is configured to reduce the cross-silicon data transmission rate or suspend transmission, and resume data transmission after the temperature returns to below the preset threshold.

[0082] Based on the aforementioned tiered rate limiting, this invention further provides two optional processing strategies. If a certain aggregation router is already transmitting due to temperature restrictions, depending on the data's quality of service requirements, it has two optional processing methods: First, reduce the transmission frequency or even stop transmitting, waiting for the temperature to drop to the threshold temperature before resuming full-speed transmission; Second, based on the known congestion and temperature status information of other nearby nodes, forward the data packets to another aggregation router that is not under temperature control and rate limiting and is not congested, which will then act as a proxy for the cross-die transmission of the data packets (hereinafter referred to as the proxy aggregation router or proxy router). The congestion and temperature status information comes from real-time synchronization of the tree control network.

[0083] It should be noted that the proxy aggregation router configures an independent and exclusive virtual channel (VC) for proxy transmissions to carry data packets forwarded by the proxy. This independent and exclusive virtual channel is used to establish resource isolation between the data path used by the proxy transmission and the data path used by non-proxy transmissions in the aggregation network, so as to avoid the proxy transmission data stream and the non-proxy transmission data stream competing for the same link resources and forming a circular dependency relationship, thereby preventing deadlock.

[0084] It is important to note that without this dedicated virtual channel, proxy forwarding between aggregation routers in the vertical direction (Z-axis, i.e., the direction in which multiple dies are stacked vertically) may form a closed loop, resulting in Z-axis loop dependency and causing link deadlock.

[0085] Furthermore, the temperature control and flow control can be implemented at the protocol layer by extending the header fields of the data packets or introducing dedicated control messages. For example, temperature status identifiers or flow control level identifiers can be carried in the data packets of the aggregation network, so that each aggregation router can execute a consistent flow limiting policy based on the current temperature status during forwarding; or the bandwidth quota of cross-silicon chip links can be dynamically configured through independent control signaling.

[0086] In addition, the temperature and flow control mechanism can be jointly optimized by combining the physical characteristics of cross-silicon links (such as TSV bandwidth, latency and power consumption characteristics). Under the premise of ensuring the critical data transmission requirements, adaptive throttling is implemented for non-critical data streams to reduce heat accumulation in hot spots.

[0087] The temperature status information and / or flow control information required for temperature and flow control are transmitted through the control network to achieve load information synchronization and coordinated scheduling among the three types of networks. Specifically, the temperature status information may originate from temperature sensing units located near each silicon wafer or each aggregation router. The temperature sensing units periodically or on demand collect temperature data and upload it to the intermediate or root node of the control network for aggregation and analysis. The root or intermediate node of the control network can generate corresponding flow control information based on the global or local temperature distribution and distribute it to each aggregation router through the control network.

[0088] Besides temperature status information, the control network is also responsible for synchronizing and distributing congestion status information. The role of the control layer router in the entire path planning and flow control process is to synchronize congestion status information between aggregation routers and from PEs to aggregation routers. For example, if an aggregation router experiences congestion, it will send a broadcast message to notify all other aggregation routers and PEs on the same die. Upon receiving the congestion message, other aggregation routers will temporarily suspend forwarding proxy requests to that aggregation router. After the PE integrates the congestion status information from all aggregation routers, it will backpressure the traffic that needs to be transmitted across dies before the protocol layer reverses. Similarly, a broadcast message will be sent to notify the network when congestion is resolved.

[0089] Furthermore, the flow control information may include cross-wafer link bandwidth limit parameters, data injection rate limit, priority adjustment strategy, and flow limiting switch signal, etc. After receiving the flow control information, each aggregation router updates its corresponding cross-wafer transmission behavior, thereby realizing a globally consistent or regionally differentiated temperature and flow control strategy.

[0090] Through the above mechanism, the control network is responsible for the rapid distribution and synchronization of temperature status information and control commands, while the aggregation network is responsible for specific data flow limiting and path scheduling execution, thus forming a closed-loop control process of perception, decision-making and execution. This enables the main data network, aggregation network and control network to achieve collaborative scheduling based on unified load and temperature status, effectively suppressing the thermal accumulation problem in 3D stacked chips while ensuring system performance.

[0091] To more intuitively illustrate the data packet transmission path selection and the proxy forwarding mechanism under temperature and flow control in this invention, a specific embodiment is described below.

[0092] As an optional embodiment, taking a 3D chip NoC with a main data network size of 4×4, a subnet size of 2×2, and 4 dies stacked, the following naming rules are defined for routers with two Mesh structures: type(x, y, die). M represents the main data path router and A represents the aggregation router. For example, M(2, 3, 1) represents the main data router with the first die located at two-dimensional position (2, 3), and A(1, 1, 2) represents the aggregation router located at two-dimensional position (1, 1). The number of the two-dimensional position depends on the network type. Taking A(1, 1, 2) as an example, since the subnet dimension and subnet size are 2×2, A(1, 1, 2) is connected to A(1, 2, 2), A(2, 1, 2) and M(1, 1, 2), M(1, 2, 2), M(2, 1, 2), M(2, 2, 2) within the same die. A(1, 1, 2) is connected to A(1, 1, 1) and A(1, 1, 3) across the die.

[0093] Intra-Die Transmission: If M(1, 1, 1) wants to send data to M(3, 4, 1), there are two possibilities: 1) Transmission via the main data path, where the data is transmitted sequentially via M(1, 1, 1), M(2, 1, 1), M(3, 1, 1), M(3, 2, 1), M(3, 3, 1), and M(3, 4, 1) to the destination; 2) Transmission via the aggregation path, where the data is transmitted sequentially via M(1, 1, 1), A(1, 1, 1), A(2, 1, 1), A(2, 2, 1), and M(3, 4, 1) to the destination, reducing the number of hops and avoiding congestion on the main data path.

[0094] Inter-Die Transmission: If M(1, 1, 1) wants to send data to M(3, 4, 3), 1) if there is no overheating or congestion along the entire path, the transmission path should be M(1, 1, 1), A(1, 1, 1), A(1, 1, 2), A(1, 1, 3), A(2, 1, 3), A(2, 2, 3), M(3, 4, 3); 2) if overheating and rate limiting occur, for example, when A(1, 1, 2) forwards to A(1, 1, 3), A(1, 1, 2) cannot forward due to TSV rate limiting at this point, then A(1, 1, 2) can select a proxy router based on the congestion status of the aggregation router in the same die, for example, selecting A(1, 2, 2). In this case, A(1, 1, 2) can then send data to M(3, 4, 3). 2) Data that needs to be transmitted across dies will be marked as congestion forwarding data and forwarded to the proxy exclusive virtual channel (VC) of A(1, 2, 2). After that, the data packet will continue to be transmitted in the proxy exclusive virtual channel (VC) until the cross-silicon forwarding is completed, and then finally reach the destination via A(1, 2, 2), A(1, 2, 3), A(2, 2, 3), M(3, 4, 3).

[0095] Exemplary methods Accordingly, this invention also provides a data transmission method for a heterogeneous on-chip network of a 3D stacked chip. Applied to a 3D stacked chip comprising multiple vertically stacked silicon wafers, the heterogeneous on-chip network is disposed within each of the silicon wafers. The heterogeneous on-chip network includes a main data network and a convergence network. The main data network includes multiple main data routers, each connected to at least one processing unit. The convergence network includes multiple convergence routers, each connected to a subnet composed of multiple main data routers. The convergence routers are interconnected and connected to convergence routers of adjacent silicon wafers via a cross-wafer interconnect structure. Figure 3 This is a flowchart of a data transmission method for a heterogeneous on-chip network of a 3D stacked chip provided by an embodiment of the present invention. This embodiment includes the following steps: S301: Data is transmitted between processing units connected to the main data router through the main data router in the main data network; S302: Data is transmitted between subnets consisting of multiple primary data routers connected to the aggregation router in the aggregation network, and between the interconnected aggregation routers, as well as with aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

[0096] The heterogeneous on-chip network also includes a control network; The control network is independent of the main data network and the aggregation network in terms of physical links and protocol paths, and does not carry data transmission; The control network includes a root node, intermediate nodes, and terminal nodes. The terminal nodes of the control network are connected to the processing unit and the corresponding master data router. The intermediate nodes are connected to the aggregation router, and the root node has cross-silicon chip communication capability. The method further includes: Control signals, configuration commands, and synchronization signals are transmitted through the control network. The specific steps for transmitting control signals, configuration commands, and synchronization signals through the control network are as follows: Signals are transmitted between the processing unit and the corresponding main data router through the final node of the control network. Signals are transmitted between the aggregation routers through the intermediate nodes of the control network; Cross-wafer communication is performed through the root node of the control network.

[0097] The method further includes: Temperature and flow control for cross-silicon wafer transmission is performed at the protocol layer through the convergence network, and the cross-silicon wafer data transmission rate is actively limited based on the silicon wafer temperature. When the aggregation router is restricted from transmitting data due to a temperature exceeding a preset threshold, the aggregation router is configured to reduce the cross-silicon data transmission rate or suspend transmission, and resume data transmission after the temperature returns to below the preset threshold.

[0098] The specific steps of temperature and flow control for cross-silicon wafer transmission at the protocol layer through the aggregation network, which actively limits the cross-silicon wafer data transmission rate based on the wafer temperature, are as follows: When the aggregation router is unable to send cross-wafer data due to temperature limitations, the aggregation router forwards the data packets to be transmitted across the silicon wafer to another aggregation router that is not in a congested state and is not in a temperature-controlled current-limiting state, based on the congestion status information and temperature status information of other aggregation routers synchronized in the control network. The other aggregation router then acts as a proxy aggregation router to complete the cross-wafer transmission.

[0099] The proxy aggregation router configures an independent and exclusive virtual channel for the proxy transmission to carry data packets forwarded by the proxy; The virtual channel is configured to establish resource isolation between the data path used by the proxy transmission and the data path used by the non-proxy transmission in the aggregation network.

[0100] The temperature status information and / or flow control information required for temperature and flow control are transmitted through the control network.

[0101] The method further includes: Once a data packet enters the aggregation network from the main data network, its transmission path is restricted to the aggregation network and the cross-silicon interconnect structure, and it does not return to the main data network.

[0102] The method further includes: When the destination address of the data packet to be transmitted is within the same silicon wafer but across subnets, the master data router determines whether to forward the data packet to the aggregation router for accelerated transmission based on the quality of service requirements, data path hop count, and / or the congestion status of the aggregation network.

[0103] The multiple master data routers in the master data network constitute multiple master router clusters, each master router cluster containing a single master data router, or containing at least two master data routers stacked vertically.

[0104] The specific steps for transmitting control signals, configuration commands, and synchronization signals through the control network are as follows: The control network synchronizes protocol-level credit information and sends a backpressure signal to the upstream requesting producer when network congestion is detected.

[0105] The convergence ratio of the root node of the control network is adjusted according to the traffic load.

[0106] The main data network and / or the aggregation network adopts a mesh topology, which includes a two-dimensional mesh topology or a toroidal topology.

[0107] The control network adopts a hierarchical distribution topology, which includes tree topology, tree-like topology, or ring broadcast topology.

[0108] The primary data routers in the primary data network employ either a credit-based flow control mechanism or a handshake-based flow control mechanism.

[0109] The main data network and / or the aggregation network employ a static routing algorithm or a congestion-aware adaptive routing algorithm.

[0110] The cross-silicon interconnect structure includes through-silicon vias, silicon interposers, or switchable intermediate chips.

[0111] The subnet connected to each aggregation router in the aggregation network consists of N×M main data routers, where N and M are positive integers and N×M≥2; The aggregation router establishes a communication connection with the processing unit in its corresponding subnet through its network interface.

[0112] It should be noted that although several devices, units, or modules of heterogeneous on-chip networks for 3D stacked chips have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0113] Furthermore, although the operation of the data transmission method of the heterogeneous on-chip network of 3D stacked chips of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0114] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0115] This invention provides: 1. A heterogeneous on-chip network for a 3D stacked chip, characterized in that the 3D stacked chip comprises multiple vertically stacked silicon wafers, the heterogeneous on-chip network is disposed within each of the silicon wafers, and the heterogeneous on-chip network comprises a main data network and a convergence network; wherein... The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

[0116] 2. The heterogeneous on-chip network according to item 1, characterized in that the heterogeneous on-chip network further includes a control network; The control network is independent of the main data network and the aggregation network in terms of physical links and protocol paths, and does not carry data transmission. The control network is used to transmit control signals, configuration instructions and synchronization signals. The control network includes a root node, intermediate nodes, and terminal nodes. The terminal nodes of the control network are connected to the processing unit and the corresponding master data router, the intermediate nodes are connected to the aggregation router, and the root node has cross-silicon chip communication capability.

[0117] 3. The heterogeneous on-chip network according to item 2, characterized in that the convergence network is further configured to implement temperature-controlled flow control for cross-silicon wafer transmission at the protocol layer, and actively limit the cross-silicon wafer data transmission rate according to the silicon wafer temperature; When the aggregation router is restricted from transmitting data due to a temperature exceeding a preset threshold, the aggregation router is configured to reduce the cross-silicon data transmission rate or suspend transmission, and resume data transmission after the temperature returns to below the preset threshold.

[0118] 4. According to the heterogeneous on-chip network described in item 3, the feature is that when the aggregation router is unable to transmit data across silicon wafers due to temperature limitations, the aggregation router is configured to forward the data packets to be transmitted across silicon wafers to another aggregation router that is not in a congested state and is not in a temperature-controlled current-limiting state, based on the congestion status information and temperature status information of other aggregation routers synchronized by the control network, and the other aggregation router acts as a proxy aggregation router to complete the cross-silicon wafer transmission.

[0119] 5. The heterogeneous on-chip network according to item 4, characterized in that the proxy aggregation router configures an independent and exclusive virtual channel for the proxy transmission, used to carry data packets forwarded by the proxy; The virtual channel is configured to establish resource isolation between the data path used by the proxy transmission and the data path used by the non-proxy transmission in the aggregation network.

[0120] 6. The heterogeneous on-chip network according to item 3, characterized in that the temperature status information and / or flow control information required for the temperature control and flow control are transmitted through the control network.

[0121] 7. The heterogeneous on-chip network for temperature status information according to item 1, characterized in that, once a data packet enters the aggregation network from the main data network, its transmission path is restricted to only within the aggregation network and the cross-silicon interconnect structure, and it no longer returns to the main data network.

[0122] 8. The heterogeneous on-chip network according to item 1, characterized in that the main data router is configured to: when the target address of the data packet to be transmitted is located within the same silicon chip but across subnets, determine whether to forward the data packet to the aggregation router for accelerated transmission based on the quality of service requirements, data path hop count and / or the congestion status of the aggregation network.

[0123] 9. The heterogeneous on-chip network according to item 1, characterized in that the multiple main data routers in the main data network constitute multiple main router clusters, each main router cluster containing a single main data router, or containing at least two main data routers stacked in a vertical direction.

[0124] 10. The heterogeneous on-chip network according to item 2, characterized in that the control network is further configured to synchronize protocol-level credit information and send a backpressure signal to the upstream production end when network congestion is detected.

[0125] 11. The heterogeneous on-chip network according to item 2, characterized in that the convergence ratio of the root node of the control network is configured to be adjusted according to the traffic load.

[0126] 12. The heterogeneous on-chip network according to item 1, characterized in that the main data network and / or the aggregation network adopt a mesh topology, wherein the mesh topology includes a two-dimensional mesh topology or a toroidal topology.

[0127] 13. The heterogeneous on-chip network according to item 2, characterized in that the control network adopts a hierarchical distribution topology, the hierarchical distribution topology including tree topology, tree-like topology or ring broadcast topology.

[0128] 14. The heterogeneous on-chip network according to item 1, characterized in that the main data routers in the main data network adopt a credit-based flow control mechanism or a handshake-based flow control mechanism.

[0129] 15. The heterogeneous on-chip network according to item 1, characterized in that the main data network and / or the aggregation network adopt a static routing algorithm or a congestion-aware adaptive routing algorithm.

[0130] 16. The heterogeneous on-chip network according to item 1, wherein the cross-silicon interconnect structure includes through-silicon vias, silicon interposers, or switching intermediate chips.

[0131] 17. The heterogeneous on-chip network according to item 1, characterized in that the subnet connected to each aggregation router in the aggregation network consists of N×M main data routers, where N and M are positive integers and N×M≥2; The aggregation router establishes a communication connection with the processing unit in its corresponding subnet through its network interface.

[0132] 18. A data transmission method for a heterogeneous on-chip network in a 3D stacked chip, characterized in that it is applied to a 3D stacked chip comprising multiple vertically stacked silicon wafers, wherein the heterogeneous on-chip network is disposed within each of the silicon wafers, and the heterogeneous on-chip network includes a main data network and a convergence network; wherein The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected with each other and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure. The method includes: Data is transmitted between processing units connected to the main data router through the main data router in the main data network; Data is transmitted between subnets consisting of multiple primary data routers connected to the aggregation router in the aggregation network, and between the interconnected aggregation routers, as well as with aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

[0133] 19. The data transmission method of the heterogeneous on-chip network according to item 18, wherein the heterogeneous on-chip network further includes a control network; The control network is independent of the main data network and the aggregation network in terms of physical links and protocol paths, and does not carry data transmission; The control network includes a root node, intermediate nodes, and terminal nodes. The terminal nodes of the control network are connected to the processing unit and the corresponding master data router. The intermediate nodes are connected to the aggregation router, and the root node has cross-silicon chip communication capability. The method further includes: Control signals, configuration commands, and synchronization signals are transmitted through the control network. The specific steps for transmitting control signals, configuration commands, and synchronization signals through the control network are as follows: Signals are transmitted between the processing unit and the corresponding main data router through the final node of the control network. Signals are transmitted between the aggregation routers through the intermediate nodes of the control network; Cross-wafer communication is performed through the root node of the control network.

[0134] 20. The data transmission method for a heterogeneous on-chip network according to claim 19, characterized in that the method further comprises: Temperature and flow control for cross-silicon wafer transmission is performed at the protocol layer through the convergence network, and the cross-silicon wafer data transmission rate is actively limited based on the silicon wafer temperature. When the aggregation router is restricted from transmitting data due to a temperature exceeding a preset threshold, the aggregation router is configured to reduce the cross-silicon data transmission rate or suspend transmission, and resume data transmission after the temperature returns to below the preset threshold.

[0135] 21. The data transmission method for heterogeneous on-chip networks according to item 20, characterized in that the step of performing temperature and flow control for cross-silicon wafer transmission at the protocol layer through the aggregation network, and actively limiting the cross-silicon wafer data transmission rate based on the silicon wafer temperature, specifically comprises: When the aggregation router is unable to send cross-wafer data due to temperature limitations, the aggregation router forwards the data packets to be transmitted across the silicon wafer to another aggregation router that is not in a congested state and is not in a temperature-controlled current-limiting state, based on the congestion status information and temperature status information of other aggregation routers synchronized in the control network. The other aggregation router then acts as a proxy aggregation router to complete the cross-wafer transmission.

[0136] 22. The data transmission method for heterogeneous on-chip networks according to item 21, characterized in that the proxy aggregation router configures an independent and exclusive virtual channel for the proxy transmission to carry data packets forwarded by the proxy; The virtual channel is configured to establish resource isolation between the data path used by the proxy transmission and the data path used by the non-proxy transmission in the aggregation network.

[0137] 23. The data transmission method of the heterogeneous on-chip network according to item 20, characterized in that the temperature status information and / or flow control information required for the temperature control and flow control are transmitted through the control network.

[0138] 24. The data transmission method for heterogeneous on-chip networks according to claim 18, characterized in that the method further includes: Once a data packet enters the aggregation network from the main data network, its transmission path is restricted to the aggregation network and the cross-silicon interconnect structure, and it does not return to the main data network.

[0139] 25. The data transmission method for a heterogeneous on-chip network according to claim 18, characterized in that the method further includes: When the destination address of the data packet to be transmitted is within the same silicon wafer but across subnets, the master data router determines whether to forward the data packet to the aggregation router for accelerated transmission based on the quality of service requirements, data path hop count, and / or the congestion status of the aggregation network.

[0140] 26. The data transmission method of the heterogeneous on-chip network according to item 18, characterized in that the multiple main data routers in the main data network constitute multiple main router clusters, each main router cluster containing a single main data router, or containing at least two main data routers stacked in a vertical direction.

[0141] 27. The data transmission method for heterogeneous on-chip networks according to item 19, characterized in that the step of transmitting control signals, configuration instructions, and synchronization signals through the control network specifically comprises: The control network synchronizes protocol-level credit information and sends a backpressure signal to the upstream requesting producer when network congestion is detected.

[0142] 28. The data transmission method of the heterogeneous on-chip network according to item 19, characterized in that the convergence ratio of the root node of the control network is adjusted according to the traffic load.

[0143] 29. The data transmission method of the heterogeneous on-chip network according to item 18, characterized in that the main data network and / or the aggregation network adopts a mesh topology, wherein the mesh topology includes a two-dimensional mesh topology or a toroidal topology.

[0144] 30. The data transmission method for heterogeneous on-chip networks according to item 19, characterized in that the control network adopts a hierarchical distribution topology, the hierarchical distribution topology including tree topology, tree-like topology or ring broadcast topology.

[0145] 31. The data transmission method of the heterogeneous on-chip network according to item 18, characterized in that the main data routers in the main data network adopt a credit-based flow control mechanism or a handshake-based flow control mechanism.

[0146] 32. The data transmission method for heterogeneous on-chip networks according to item 18, characterized in that the main data network and / or the aggregation network adopt a static routing algorithm or a congestion-aware adaptive routing algorithm.

[0147] 33. The data transmission method of heterogeneous on-chip network according to item 18, characterized in that the cross-silicon interconnect structure includes through-silicon vias, silicon interposers, or switching intermediate chips.

[0148] 34. The data transmission method of the heterogeneous on-chip network according to item 18, characterized in that the subnet connected to each aggregation router in the aggregation network is composed of N×M main data routers, where N and M are positive integers and N×M≥2; The aggregation router establishes a communication connection with the processing unit in its corresponding subnet through its network interface.

Claims

1. A heterogeneous on-chip network for 3D stacked chips, characterized in that, The 3D stacked chip includes multiple vertically stacked silicon wafers, and the heterogeneous on-chip network is disposed within each of the silicon wafers. The heterogeneous on-chip network includes a main data network and a convergence network. in The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

2. The heterogeneous on-chip network according to claim 1, characterized in that, The heterogeneous on-chip network also includes a control network; The control network is independent of the main data network and the aggregation network in terms of physical links and protocol paths, and does not carry data transmission. The control network is used to transmit control signals, configuration instructions and synchronization signals. The control network includes a root node, intermediate nodes, and terminal nodes. The terminal nodes of the control network are connected to the processing unit and the corresponding master data router, the intermediate nodes are connected to the aggregation router, and the root node has cross-silicon chip communication capability.

3. The heterogeneous on-chip network according to claim 2, characterized in that, The aggregation network is also configured to implement temperature and flow control for cross-wafer transmission at the protocol layer, actively limiting the cross-wafer data transmission rate based on the wafer temperature. When the aggregation router is restricted from transmitting data due to a temperature exceeding a preset threshold, the aggregation router is configured to reduce the cross-silicon data transmission rate or suspend transmission, and resume data transmission after the temperature returns to below the preset threshold.

4. The heterogeneous on-chip network according to claim 3, characterized in that, When the aggregation router is unable to transmit data across silicon wafers due to temperature limitations, the aggregation router is configured to forward the data packets to be transmitted across silicon wafers to another aggregation router that is not in a congested state and is not in a temperature-controlled current-limiting state, based on the congestion status information and temperature status information of other aggregation routers synchronized by the control network. The other aggregation router then acts as a proxy aggregation router to complete the cross-silicon wafer transmission.

5. The heterogeneous on-chip network according to claim 4, characterized in that, The proxy aggregation router configures an independent and exclusive virtual channel for the proxy transmission to carry data packets forwarded by the proxy; The virtual channel is configured to establish resource isolation between the data path used by the proxy transmission and the data path used by the non-proxy transmission in the aggregation network.

6. The heterogeneous on-chip network according to claim 3, characterized in that, The temperature status information and / or flow control information required for temperature and flow control are transmitted through the control network.

7. The heterogeneous on-chip network for temperature state information according to claim 1, characterized in that, Once a data packet enters the aggregation network from the main data network, its transmission path is restricted to the aggregation network and the cross-silicon interconnect structure, and it does not return to the main data network.

8. The heterogeneous on-chip network according to claim 1, characterized in that, The primary data router is configured to: when the destination address of the data packet to be transmitted is within the same silicon wafer but across subnets, determine whether to forward the data packet to the aggregation router for accelerated transmission based on the quality of service requirements, data path hop count and / or the congestion status of the aggregation network.

9. The heterogeneous on-chip network according to claim 1, characterized in that, The multiple master data routers in the master data network constitute multiple master router clusters, each master router cluster containing a single master data router, or containing at least two master data routers stacked vertically.

10. A data transmission method for a heterogeneous on-chip network of a 3D stacked chip, characterized in that, Applied to a 3D stacked chip comprising multiple vertically stacked silicon wafers, wherein the heterogeneous on-chip network is disposed within each of the silicon wafers, and the heterogeneous on-chip network includes a main data network and a convergence network; in The main data network includes multiple main data routers, each of which is connected to at least one processing unit; The aggregation network includes multiple aggregation routers, each of which is connected to a subnet consisting of multiple master data routers. The aggregation routers are interconnected with each other and connected to aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure. The method includes: Data is transmitted between processing units connected to the main data router through the main data router in the main data network; Data is transmitted between subnets consisting of multiple primary data routers connected to the aggregation router in the aggregation network, and between the interconnected aggregation routers, as well as with aggregation routers on adjacent silicon wafers through a cross-wafer interconnect structure.

Images