A method of forwarding and returning data packets in a multi-die system on chip (SOC)

Abstract

The invention relates to a method (100) for forwarding and returning data packets in a multi-die system on chip (SoC). The method (100) is characterized by the steps of forwarding a request data packet from a source die to a destination die (110), and returning a response data packet from the destination die to the source die (120). The method (100) utilizes track IDs for track sharing and utilizes a departure port count to track the number of inter-die ports a data packet has passed through, thereby ensuring efficient routing and enhancing die communication between multiple dies within the system.

Description

Technical Field

[0001] This invention generally relates to multi-die system-on-a-chip (SoC), and more specifically, to a method for routing inter-die communication within a network-on-a-chip (NoC) of a multi-die SoC. Background Technology

[0002] As semiconductor technology evolves towards increasingly complex multi-die SoC designs, including those for heterogeneous computing and artificial intelligence (AI) accelerators, traditional routing methods have proven unscalable. In traditional single-die systems, memory requests are managed using explicit memory bases and memory-limited addressing techniques, requiring numerous routing table lookups. However, when scaling to multi-die architectures, the number of lookup entries for multiple source-destination dies grows exponentially, complicating routing table management and increasing latency. To address these challenges, many SoCs are adopting multi-die strategies, breaking down a monolithic SoC into smaller dies interconnected using inter-die connectivity IPs such as UCIe. This shift necessitates efficient on-chip network (Noc) management across multiple dies, presenting key technical challenges. Two main issues arise in inter-die NoC management: enabling die-to-die communication via inter-die interconnects such as UCIe without unifying die configurations, and effectively managing routing tracks across multiple silicon dies to reduce latency and complexity. In response, various methods and systems have been developed to improve existing NoC in multi-die systems, some of which are discussed in the following prior art references: US Patent Publication No. US2019227963A1 discloses an integrated circuit system designed for high-bandwidth, low-latency interconnects for inter-die or intra-die communication. The system includes a first die with structured circuit sectors, a second die with modular peripheral intellectual property (IP), a passive silicon inserter coupling the first die to the second die, and a modular interface including a network on-chip (NOC). The modular interface provides high-bandwidth, low-latency communication between the first and second dies, between structured circuit sectors, and between the first and third dies.

[0003] U.S. Patent No. 11,496,418 discloses a data transmission method for an integrated circuit having a NoC capable of packet-based and time-multiplexed communication. The method includes providing a router network coupled to multiple bridging circuits, each bridging circuit connected to a client circuit. The bridging circuits provide a packet-based interface to the client circuits connected thereto, enabling them to send and receive packet flits of transactions initiated by the client circuits. The bridging circuits perform sending and receiving according to a predetermined schedule. The method also includes simultaneously transmitting packet flits corresponding to different client circuits using time multiplexing on a shared physical channel of the router network.

[0004] While these references represent advancements in inter-die communication, certain limitations remain. Notably, none of these references specifically address the challenges associated with managing routing tracks as the number of interconnected dies increases, inevitably leading to an expansion of routing table entries. Furthermore, these references lack mechanisms for accurately monitoring packet traversal across dies, which is crucial for maintaining data integrity and improving communication efficiency. Summary of the Invention

[0005] The following is a simplified overview of the invention to provide a basic understanding of certain aspects of it. This overview is not a comprehensive summary of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that follows.

[0006] The purpose of this invention is to provide a method for forwarding and returning packets in a multi-die system-on-a-chip (SoC) that ensures efficient management of routing tracks across multiple dies, thereby enhancing inter-die communication within the system.

[0007] Another object of the present invention is to provide a method for forwarding and returning data packets in a multi-die system-on-a-chip (SoC) that ensures efficient management of routing tracks across multiple dies without requiring uniform die configuration.

[0008] Therefore, these objectives can be achieved by following the teachings of this invention. This invention relates to a method for forwarding and returning data packets in a multi-die system-on-a-chip (SoC), characterized by steps A) forwarding a request packet from a source die to a destination die, and B) returning a response packet from the destination die to the source die. Step A includes the following steps: performing a first lookup to determine a forwarding routing path for the request packet based on a memory base and a memory limit range; identifying and assigning a routing track to the request packet during the first lookup, the routing track being used to route the corresponding response packet back to the source die; initializing a Departure Port Count (DPC) to track the number of inter-die ports traversed by the request packet along the forwarding path, wherein the DPC increments each time the packet traverses an inter-die port on the forwarding path; transmitting the request packet from the source die to a first inter-die port; and performing a subsequent lookup after each inter-die port to transmit the request packet to the destination die. Step B includes the following steps: identifying the routing track assigned during the first lookup; transmitting the response packet from the destination die to the inter-die port identified in the routing track; transmitting the response packet to the source die through one or more inter-die ports according to the routing track; updating the DPC by decrementing it each time the response packet passes through an inter-die port on the return path; when the DPC reaches zero, performing a return lookup at the source die using the source ID; and transmitting the response packet to the source ID.

[0009] The present invention comprises a combination of features and components fully described and illustrated in the accompanying drawings. It should be understood that various changes to the details may be made without departing from the scope of the invention or sacrificing any of its advantages. Attached Figure Description

[0010] To gain a detailed understanding of the above-described features of the invention, a more specific description of the invention briefly summarized above can be referenced through embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical embodiments of the invention and should not be construed as limiting its scope. Other equally effective embodiments are permissible with respect to the invention.

[0011] These and other features, benefits, and advantages of the invention will become apparent from the following figures, wherein like reference numerals denote like structures in the views, wherein: Figure 1 This is a flowchart illustrating a method for forwarding and returning data packets in a multi-die on-chip system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating step A of forwarding a request packet from a source die to a destination die according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating step B of returning a response data packet from the destination die to the source die according to an embodiment of the present invention; Figure 4 These are images illustrating a general pre-planned routing track and a custom routing track in a multi-die on-chip system according to an embodiment of the present invention; Figure 5 This is an image illustrating a request data packet for a silicon 3 device 4 in a multi-die on-chip system according to an embodiment of the present invention; and Figure 6 This is an image illustrating a response data packet returned from silicon device 4 in a multi-die on-chip system according to an embodiment of the present invention. Detailed Implementation

[0012] Detailed embodiments of the invention are disclosed herein as needed; however, it should be understood that the disclosed embodiments are merely examples of the invention, which can be embodied in various forms. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as the basis for the claims. It should be understood that the accompanying drawings and their detailed description are not intended to limit the invention to the specific forms disclosed; rather, the invention will cover all modifications, equivalents, and alternatives falling within the scope of the invention as defined in the appended claims. The word “may” as used in this application is used in a permissible sense (i.e., meaning possible), not in a mandatory sense (i.e., meaning mandatory). Similarly, the words “include,” “including,” and “includes” mean including but not limited to. Furthermore, unless otherwise stated, the words “a” or “an” mean “at least one,” and the word “plurality” means one or more. Abbreviations or technical terms used indicate meanings generally accepted in the art.

[0013] The invention is described below with reference to the accompanying drawings, which correspond to the same elements throughout the description. However, the invention can be embodied in many different forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to make this disclosure thorough and complete, and to fully convey the scope of the invention to those skilled in the art. In the following detailed description, numerical values ​​and ranges are provided for various aspects of the described implementations. These values ​​and ranges are by way of example only and are not intended to limit the scope of the claims. Furthermore, many materials are identified as applicable to various aspects of the implementation. These materials are to be considered exemplary and are not intended to limit the scope of the invention.

[0014] Reference Figures 1 to 4The invention will now be described in more detail, based on the accompanying drawings.

[0015] This invention relates to a method (100) for forwarding and returning data packets in a multi-die system-on-a-chip (SoC), comprising the steps of forwarding a request packet from a source die to a destination die (110), and then returning a response packet from the destination die to the source die (120), as follows. Figure 1 As shown. In this specification, the term "source-destination pair" refers to the relationship between a source die that initiates a request and its corresponding destination die.

[0016] refer to Figure 2 This illustrates the steps for forwarding a request packet from a source die to a destination die (110). The steps for forwarding a request packet from the source node of the source die to the destination node (110) at the destination die include the following sub-steps: (11) performing a first lookup to determine a forwarding route for the request packet based on the memory base and memory limit range of the destination; (12) identifying and assigning a route track to the request packet during the first lookup, the route track being used to route the response packet back to the source die; (13) initializing a departure port count (DPC) to track the number of inter-die (1) ports that the request packet has traversed along the forwarding path, wherein the DPC increments each time the packet traverses an inter-die (1) port on the forwarding path; (14) transferring the request packet from the source die to the first inter-die (1) port; and (15) performing a subsequent lookup after each inter-die (1) port to transfer the request packet to the destination die.

[0017] In step 11, the request packet performs a first lookup at the source die to determine a forwarding route. The request packet can be a memory read request and / or memory write request originating from a source node at the source die, targeting a destination die. The source node can be one of multiple cores at the source die. The first lookup performed by the request packet is based on the memory base and memory limit range of the target destination node. The memory base and memory limit range define specific ranges of memory addresses within the system. The memory base refers to the lower memory range, while the memory limit refers to the higher memory range. This memory is used to locate devices or nodes on a multi-die SOC. This range is used in the routing table to retrieve routing information for the corresponding source-destination pair to determine the forward route path.

[0018] Source-destination pairs can be further subdivided into multiple memory bases and memory limit ranges. This fragmentation allows the routing process to distribute traffic across various paths, optimizing performance and preventing deadlocks. Furthermore, memory bases and memory limit ranges can be compressed into supersets of their respective memory bases and memory limit ranges. This compression can be applied to multiple source-destination pairs using similar routing tracks in the routing table, thereby reducing the number of entries in the lookup table. When the destination lithography is far from the lithography performing the lookup, superset memory bases and memory limit ranges can be defined in the routing table to further optimize routing table entries. There is no limit to the number of source-destination pairs that can be combined if the superset memory bases and memory limit ranges do not conflict with other destination memory lookup entries, in order to minimize routing lookups.

[0019] In step 12, during the first lookup, a routing track is identified and assigned to the request packet, whereby the routing track is used to route the corresponding response packet back to the source die. The assigned routing track is selected from a pre-established set of tracks for defining transmission paths across multiple dies, each of which is identified by a track ID. The track ID of each track corresponds to a specific return routing path across multiple dies, excluding tracks within the source and destination dies. Specifically, the track ID identifies the track on the remote die that defines the return path. Once a routing track is assigned, the request packet receives the track ID and carries it along the forwarding path to the destination die. Thus, multiple source-destination pairs using similar return routing paths can share the same track ID for routing their respective response packets back to the source die, thereby facilitating efficient communication on shared paths.

[0020] According to embodiments of the present invention, the multiple tracks include general, pre-planned cabling tracks or customized cabling tracks with routers (2), such as... Figure 4 As shown. Tracks 00 (T00) to 05 (T05) are common, pre-planned route tracks, while track 06 (T06) is as follows. Figure 4 The diagram illustrates a custom routing track. A generic, pre-planned routing track is shared by multiple source-destination pairs that use similar tracks when packets traverse dies in a multi-die on-chip system. The generic, pre-planned routing track can look up its routing information based on a superset memory base and memory constraints. Custom routing tracks, on the other hand, are suitable for source-destination pairs that require special or unique routing tracks to ensure high performance. Custom routing tracks need to be independently mapped to their own memory base and memory constraints to facilitate the lookup of their routing information.

[0021] As described in steps 11 and 12, during the first lookup, the response packet retrieval includes routing information for forwarding and returning routing paths. It will be apparent to those skilled in the art that steps 11 and 12 can be performed simultaneously or sequentially.

[0022] In step 13, the Departure Port Count (DPC) is initialized to track the number of inter-die (1) ports traversed by the request packet along the forwarding path. The DPC is used to track the number of inter-die (1) ports traversed by the request packet along the forwarding path and the number of corresponding response packets traversed along the return path. Specifically, the DPC is incremented by 1 when the request packet leaves its current die and enters the next adjacent die, and decremented by 1 when the corresponding response packet leaves its current die and arrives at the next adjacent die, eventually reaching 0 when it reaches the source die. In addition to the track ID, the DPC is also carried by the request packet as it travels along the forwarding path toward the destination die. The DPC is further enhanced to accommodate non-uniform die traversal, where the number of inter-die (1) ports traversed by the request packet from the source die to the destination die is different from the number of inter-die (1) ports traversed by the corresponding response packet returning from the destination die to the source die. This enhancement requires incorporating the DPC count into the return path lookup when a request packet is received from the inter-die (1) port, and overriding the incoming cumulative DPC count. The number of inter-die (1) ports traversed by the packet can be pre-calculated by the planning module and included as part of the return path lookup. When a response packet is on its return path, the incoming cumulative DPC count can be overridden as needed. The DPC enhancement is for non-uniform die traversal and is applicable to multi-die on-chip systems involving heterogeneous dies.

[0023] In step 14, the request packet is transmitted from the source die to the first inter-die (1) port that traverses the forwarding route path, thereby incrementing the DPC by 1. When the request packet arrives at the first inter-die (1) port, the routing information is subsequently terminated at the first inter-die (1) port.

[0024] In step 15, a follow-up lookup is performed after each inter-die (1) port to transmit the request packet to the destination die. Each time a request packet arrives at an inter-die (1) port, a new lookup based on the memory base and memory limit range is performed. This lookup retrieves routing information for the next inter-die (1) port the request packet will traverse along the forwarding path until it reaches the destination die. Depending on the SoC requirements, including performance and deadlock avoidance, the memory base and memory limit range used for routing can correspond to a specific memory range and memory limit associated with a custom routing track, or a superset of the memory base and memory limit range. The specific memory range and memory limit of the custom routing track are preferred when the request packet requires optimized routing for improved performance.

[0025] When a request packet arrives at the destination die, local routing is performed by looking up the memory base and limits to deliver the request packet to the destination node or device. The superset memory base and limits will not be used on the destination die.

[0026] refer to Figure 3 The diagram illustrates the steps (120) of returning a corresponding response packet from the destination die to the source die. The step (120) of returning a response packet from the destination die to the source die includes the following sub-steps: (21) identifying the routing track assigned during the first lookup; (22) transmitting the response packet within the destination die to the inter-die (1) port identified in the routing track; (23) transmitting the response packet to the source die through one or more inter-die (1) ports according to the routing track; (24) updating the DPC by decrementing the DPC each time the response packet passes through an inter-die (1) port on the return path; (25) performing a return lookup at the source die using the source ID when the DPC reaches zero; and (26) transmitting the response packet to the source ID.

[0027] In step 21, the route track assigned during the first lookup is identified by the response packet in the destination die using a track ID. The assigned route track is the return path of the response packet from the destination die to the source die. However, within the destination die where the response was generated, the assigned route track is not directly used for routing. Instead, the response packet uses the track ID of the specified route track to locate the inter-die (1) port within the specified route track in order to transmit the packet to the inter-die (1) port to begin routing on the route track.

[0028] In step 22, the response packet is transmitted from the destination node to the inter-die (1) port identified in the assigned routing track.

[0029] In step 23, the response packet is then transmitted to the source die via one or more inter-die (1) ports according to the assigned routing track. Each time the response packet traverses the die via the inter-die (1) port, it uses the track ID to look up the assigned routing track.

[0030] In step 24, the DPC is updated by decrementing it each time a response packet passes through the inter-die (1) port on the return path. Specifically, the DPC is decremented by 1 each time a response packet passes through the die.

[0031] In step 25, when the DPC reaches 0, a return lookup is performed at the source die using the source ID, signaling that the response packet has arrived at the source die. The source ID represents the identifier of the source die where the request packet was initially initiated. Within the source die, the assigned routing track is not used to route the response packet back to the source die. Instead, the response packet uses the source ID to determine the path back to the source die.

[0032] In step 26, the response data packet is transmitted to the source node along the return path identified by the source ID.

[0033] The method of the present invention for forwarding and returning data packets in a multi-die system-on-a-chip (100) enhances inter-die communication across dies by utilizing a track ID for track sharing and a DPC for tracking the number of inter-die (1) ports traversed by the data packets. By using track IDs for routing instead of relying on the original source IDs, the method (100) allows multiple source-destination pairs to share the same routing entries. This eliminates the need to unique source IDs for duplicate dies, enabling inter-die communication via inter-die interconnects such as UCle. Since multiple source-destination pairs are routed based on the same routing entries, the method (100) is able to compress the routing table within each local die, preventing the routing table entries from growing exponentially as more dies are interconnected. Furthermore, the method (100) supports heterogeneous die connections, allowing each die to be developed independently without needing to know the internal details of other dies. The use of DPC in the method (100) further facilitates accurate packet tracking. It further distinguishes when a request packet leaves its source die and moves to the next adjacent die, when a response packet moves to the next adjacent die in the destination die, and when a response packet arrives at the source die. This ensures data integrity throughout the multi-die system.

[0034] In the following text, reference will be made to Figure 5 and Figure 6 Examples of the invention are provided for more detailed illustration. These examples make it easier to understand the advantages of the invention and put it into practice. However, it should be understood that the following examples are not intended to limit the scope of the invention in any way. References Figure 4-6 The letters H, N, and D represent host, node, and device, respectively.

[0035] Example Example 1: Memory write request for Silicon 3 Device 4 (D04) Figure 5This shows that Silicon 0 Core 0 (H00) initiates a request to Silicon 3 Device 4 (D04). When the initiator's request packet looks up the routing information (RINFO) for Silicon 0, it also looks up the pre-planned return track, which in this case is track 6 (T06). The trace information (trace ID), departure port count (DPC), and routing information carried with the request packet are directed to the next adjacent die. When the initiator's request packet enters the next adjacent die (Silicon 1 in this case), the DPC counter increments by 1, and new routing information is looked up. This same process is repeated when the request packet enters Silicon 3. The track ID and DPC are carried with the request packet until it reaches its destination, Silicon 3 Device 4 (D04).

[0036] Example 2: A memory write request for silicon 3 device 4 (D04) is returned via track 6 (T06). Figure 6 The response packet returned from device 4 (D04) in Silicon 3 is shown. At the destination die, the response packet looks up the routing track for the inter-die (1) port corresponding to track 6 (T06) of the intermediate die. When the response packet in Silicon 3 is... Figure 6 When marked as "Track 6 lookup" (3), I route track lookup track ID 6.

[0037] When the response packet leaves the destination die, i.e., Silicon 3 Device 4 (D04), and heads towards the intermediate die (Silicon 1), the DPC counter decrements by 1. At the intermediate die, routing information traversing the intermediate die is retrieved using route track ID 6. This process repeats each time the response packet traverses a die, until it reaches its initiator request home die, i.e., Silicon 0, at which point the DPC counter decrements to 0. When returning to the initiator request home die, the source ID is used to look up the initiator source die, such as... Figure 6 The “Source ID Lookup” (4) is shown in the table.

[0038] For on-chip network (NOC) inter-chip solutions that pair with external inter-chip (1) ports via the AXI interface, response packets must pass through the exact same inter-chip (1) port sequence as request packets. However, for better performance or to avoid deadlock, the routing path within the chip can be different.

[0039] If the Network on-Chip (NOC) uses proprietary protocols, including the Nembus Protocol (NBSP), and is paired with inter-chip protocols such as UCIE, response packets do not have to pass through the same inter-chip (1) port, thus enabling better performance and deadlock avoidance.

[0040] The exemplary implementations described above are shown with specific features, but the scope of the invention includes a variety of other features.

[0041] Various modifications to these embodiments will be apparent to those skilled in the art from the specification and accompanying drawings. The principles associated with the various embodiments described herein can be applied to other embodiments. Therefore, this description is not intended to be limited to those described herein. Figure 1 The embodiments shown are intended to provide the broadest scope consistent with the principles, novelty, and inventiveness disclosed or suggested herein. Therefore, the invention is contemplated to reserve all other such alternatives, modifications, and variations falling within the scope of the invention and the appended claims.

[0042] It should be understood that any prior art publications cited in this article do not constitute an acknowledgment that such publications constitute part of common general knowledge in the art.

[0043] In the following claims and the preceding description of the invention, unless the context requires otherwise due to the language of expression or necessary meaning, the word “comprise” or variations such as “comprises” or “comprising” are used in an inclusive sense, that is, to specify the presence of the said feature but not to exclude the presence or addition of other features in various embodiments of the invention.

Claims

1. A method (100) for forwarding and returning data packets in a multi-die system-on-a-chip (SoC), the method (100) being characterized by the following steps: A) Forwarding the request packet from the source die to the destination die (110) includes the following steps: Perform a first lookup to determine the forwarding route path of the request packet based on the memory base and memory limit range of the target destination (11). During the first lookup, a routing track is identified and assigned to the request packet (12), the routing track being used to route the response packet back to the source die; Initialize the Departure Port Count (DPC) to track the number of inter-disk ports traversed by the request packet along the forwarding path (13), wherein the DPC is incremented each time the packet traverses an inter-disk port on the forwarding path; Transmit the request data packet from the source die to the first die-to-die port (14). A follow-up lookup is performed after each inter-die port to transmit the request packet to the destination die (15). B) Returning the response data packet from the destination die to the source die (120) includes the following steps: Identify the route track (21) assigned during the first lookup; The response data packet within the destination die is transmitted to the inter-die port (22) identified in the routing track. According to the routing track, the response data packet is transmitted to the source die (23) through one or more die-to-die ports. Each time the response data packet passes through the inter-die port on the return path, the DPC is updated by decrementing the DPC (24). When the DPC reaches zero, a return lookup is performed at the source die using the source ID (25). The response data packet is transmitted to the source ID (26).

2. The method of claim 1, wherein the assigned routing track is selected from a plurality of pre-established tracks for defining transmission paths across multiple dies, wherein, Each of the plurality of tracks is identified by a track ID.

3. The method according to claim 2, wherein, The multiple tracks include general, pre-planned routing tracks or customized routing tracks.

4. The method according to claim 1, wherein, When the response packet is returned, if the number of inter-die ports traversed by the request packet from the source to the destination is different from the number of inter-die ports traversed by the response packet from the destination back to the source, then the DPC count is overridden.

5. The method according to claim 1, wherein, The forwarding path route is determined by performing the lookup based on the memory base and the memory limit range to retrieve routing information for each source-destination pair in the routing table.

6. The method according to claim 5, wherein, The memory base and memory limit range of source-destination pairs that share similar routing tracks in the routing table are combined into a superset memory base and memory limit range.

7. The method according to claim 6, wherein, When the destination die is far from the die from which the lookup is performed, the superset memory base and memory limit range are defined in the routing table.

8. The method according to claim 1, wherein, The assigned routing track is not used to route the data packet within the source die that initiated the request or the destination die that generated the response.

9. The method according to claim 1, wherein, When the response data packet arrives at the source die, the data packet is transmitted based on the local routing information retrieved from the return lookup performed using the source ID.

Images