On-chip network tunneling method and apparatus based on UCIe flit format
By employing message token framing technology on the UCIe flit format, the NBSP flit is encapsulated into the UCIe flit format, solving the problems of inter-die communication latency and performance degradation, and realizing low-latency, high-performance on-chip network tunnel transmission, supporting efficient transmission of multiple data streams.
Patent Information
- Application Number
- CN202610631759.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-05-09
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies suffer from latency and performance degradation in inter-die communication, especially in high-performance computing systems, particularly in the field of artificial intelligence, which negatively impacts key performance indicators.
Using Message Token Framing (PTF) technology, Nembus Protocol (NBSP) flits or their fragments are encapsulated into Universal Core Interconnect High Speed (UCIe) flit format, and tunneled on the UCIe flit format. The NBSP flits are framed by PTF to identify start and end boundaries, and carry information such as message length, type and channel type to achieve cross-NoC tunnel transmission.
It significantly reduces inter-die communication latency, improves performance, reduces area usage, supports efficient transmission of consistent and inconsistent data streams, bypasses redundant protocol conversion, and optimizes system performance.
Smart Images

Figure CN123074873A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to inter-die communication. Specifically, this invention relates to a computer-implemented method and apparatus for tunneling network-on-chip (NoC) in a low-latency, high-performance manner over the Universal Interconnect-with-Chip (UCIe) flit format. Background Technology
[0002] As the demand for high-performance computing systems, especially in the field of artificial intelligence (AI), continues to increase, the complexity of these systems is also escalating. To meet the challenges of advanced process nodes, modern silicon designs employ multiple smaller dies interconnected via packaging technologies such as 2D or 2.5D interposers. Universal Chip Interconnect High Speed (UCIe) protocols provide a low-latency, high-bandwidth solution for inter-die communication bridging, and the UCIe protocol stack typically supports industry-standard protocols such as PCIe and CXL. However, converting inter-die communication to these protocols can lead to unnecessary latency, performance degradation, and inefficient area utilization, all of which negatively impact key performance indicators (KPIs). Therefore, the ability to tunnel network-on-chip (NoC) flow controllers between dies is crucial for optimizing system performance. Minimizing the latency added in inter-die communication is becoming increasingly important for meeting the stringent KPIs of modern computing systems.
[0003] US2008186998A1 describes an example of an existing method for reducing latency in NoC communication. US2008186998A1 relates to an integrated circuit containing multiple processing modules (IPs) connected by a network on-chip (NoC). Each processing module (IP) is equipped with a network interface (NI) responsible for sending data to and receiving data from the NoC. Furthermore, the document introduces a method in which at least some time slots allocated to channels originating from the same network interface are shared.
[0004] Another example is found in US2024211422A1, which discusses a multi-chip device comprising multiple ICs with interconnect NoCs. This document introduces address translation circuitry within an integrated circuit, whereby when a data stream originates from a first integrated circuit and is intended to reach its destination on a second integrated circuit, the data stream is first directed to the address translation circuitry of the second integrated circuit. This circuitry performs address translation and re-inserts the data stream into the NoC of the second IC with the appropriate destination ID.
[0005] As mentioned above, various advancements have been made to address these challenges. However, more efficient and scalable inter-die communication methods are still needed to achieve low-latency and high-performance network-on-chip (NoC) tunneling. Summary of the Invention
[0006] The following is a brief overview of the invention to provide a basic understanding of some aspects thereof. This overview is not an exhaustive summary of the invention. Its sole purpose is to present some of the concepts of the invention in a simplified form as a prelude to a more detailed description that follows.
[0007] The purpose of this invention is to provide a computer-implemented method and apparatus for tunneling network-on-chip (NoC) in a Universal Chip Interconnect High Speed (UCIe) flow control unit (flit) format with low latency and high performance.
[0008] One aspect of the present invention provides a computer-implemented method for transmission in a Network on-Chip (NoC) tunnel, characterized by the following steps: employing Packet Token Framing (PTF); encapsulating one or more Nembus Protocol (NBSP) flits or fragments thereof into a Universal Chip Interconnect High Speed (UCIe) flit format based on the available size within the UCIeflit format; appending the PTF as a packet token identifier to the NBSP flit header; framing the NBSP flit using the PTF to identify start and end boundaries, forming an encapsulated message; carrying the message length, message type, channel type, and protocol data width within the PTF; and using the encapsulated message to implement cross-NoC tunnel transmission in the UCIe flit format.
[0009] In embodiments of the present invention, each channel transmission of address writing, address reading, and write response corresponds to a single message.
[0010] In embodiments of the present invention, for write channels or read channels, the complete payload is sent as a single message in a single transmission.
[0011] In embodiments of the invention, the entire payload is combined and sent as multiple transmissions of a single message for writing to or reading from the channel.
[0012] In an embodiment of the present invention, 16 physical channels and 16 virtual channels of NBSP flit are encapsulated on the UCIe flit format.
[0013] In embodiments of the present invention, NoC tunneling based on the UCIe flit format can support both consistent and inconsistent data streams of NBSP messages.
[0014] Another aspect of the present invention provides an apparatus for Network-on-Chip (NoC) tunneling transmission, comprising a flow controller configured to encapsulate a Nembus Protocol (NBSP) flit or a fragment thereof into a Universal Chip Interconnect High Speed (UCIe) flit format based on available dimensions within the UCIe flit format; append a PTF as a message token identifier to the NBSP flit header; frame the NBSP flit using the PTF to identify start and end boundaries, forming an encapsulated message; carry message length, message type, channel type, and protocol data width in the PTF; and implement cross-NoC tunneling transmission over the UCIe flit format using the encapsulated message.
[0015] In embodiments of the present invention, message token framing can carry message payload sizes ranging from 1 data word (DW) to 1024 DWs.
[0016] The present invention consists of a combination of features and components which are fully described and illustrated below in the accompanying drawings. It should be understood that various changes may be made to the details without departing from the scope of the invention or sacrificing any of its advantages. Attached Figure Description
[0017] To further clarify various aspects of several embodiments of the present invention, the invention will be described in more detail with reference to the specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only typical embodiments of the invention and should not be considered as limiting its scope. The invention will be described and illustrated in more specific and detailed manner with reference to the following drawings, in which: Figure 1 A flowchart illustrating a computer implementation method for NoC tunnel transmission according to an embodiment of the present invention is shown; Figure 2 A block diagram of an apparatus for NoC tunnel transmission according to an embodiment of the present invention is shown; Figure 3 An example of encapsulating NBSP or AXI into two UCIe flit formats 4 according to an embodiment of the present invention is shown; Figure 4 An example of encapsulating NBSP or AXI into two UCIe flit formats 6 according to an embodiment of the present invention is shown; Figure 5 An example of NBSP protocol encapsulation with a payload message according to an embodiment of the present invention is shown; Figure 6 An example of NBSP protocol encapsulation without payload messages according to an embodiment of the present invention is shown; and Figure 7An example of a summary AMBA AXI-4 protocol payload encapsulation according to an embodiment of the present invention is shown. Detailed Implementation
[0018] While the invention has been described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the described drawings or embodiments thereof, and is not intended to represent proportions of various components. Furthermore, for ease of illustration, some components that form part of the invention may not be shown in some drawings, and such omissions do not in any way limit the embodiments outlined. It should be understood that the drawings and their detailed description are not intended to limit the invention to the specific forms disclosed, but rather, the invention covers all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims. Throughout this specification, the word “may” is used in a tolerant sense (i.e., with potential possibility) rather than in a mandatory sense (i.e., must). Furthermore, unless otherwise mentioned, the words “a” or “an” mean “at least one,” and the words “a plurality” mean “one or more.” Moreover, the terminology and expressions used herein are for descriptive purposes only and should not be construed as limiting the scope. Expressions such as “comprising,” “including,” “having,” “containing,” or “involving,” and variations thereof, are intended to have an open-ended meaning, encompassing the subject matter listed thereafter, equivalents, and additional subject matter not listed, and are not intended to exclude other additions, components, wholes, or steps. Similarly, for applicable legal purposes, the term “comprising” is considered synonymous with “including” or “containing.” Any discussions in the specification regarding documents, actions, materials, devices, articles, etc., are merely to provide background information for the invention and do not constitute, or be construed as constituting, all or part of such content as prior art or common general knowledge in the relevant field of the invention.
[0019] In this disclosure, whenever a composition, element, or group of elements is introduced by the transitional phrase “comprising,” it should be understood that the same composition, element, or group of elements is also contemplated to be preceded by the transitional phrase “consistently made of,” “comprising,” “selected from the group of,” “comprising,” or “is,” and vice versa.
[0020] The invention is described below by way of various embodiments. However, the invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, the embodiments provided make this disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numerical values and ranges are provided for various aspects of the described implementations. These numerical values and ranges should be considered as examples only and are not intended to limit the scope of the claims. Furthermore, many materials enumerated are applicable to various aspects of the implementation. These materials will be considered exemplary and are not intended to limit the scope of the invention.
[0021] This invention relates to inter-die communication. Specifically, this invention relates to a method and apparatus for tunneling on-chip networks (NoC) in a low-latency, high-performance manner over a Universal Chip Interconnect High Speed (UCIe) flow control unit (flit) format.
[0022] First refer to Figure 1 This illustrates a computer-implemented method 100 for NoC tunnel transmission according to an embodiment of the present invention. The method 100 includes the following steps: Step 102, encapsulating one or more Nembus Protocol (NBSP) flits or fragments thereof into a Universal Chip Interconnect High Speed (UCIe) flit format using token framing based on the available size within the UCIe flit format; Step 104, appending a PTF as a message token identifier to the NBSP flit header; Step 106, framing the NBSP flit using the PTF to identify start and end boundaries, forming an encapsulated message; Step 108, carrying the message length, message type, channel type, and protocol data width within the PTF; and Step 110, implementing cross-NoC tunnel transmission using the encapsulated message in the UCIe flit format.
[0023] This invention can directly encapsulate the NBSP flit format into the UCIe flit format, thereby reducing the communication latency between the local die NoC and the remote die NoC. This method bypasses the need for redundant protocol conversion, such as the Advance deXtensible Interface (AXI), and significantly reduces the area required for inter-die interconnect implementation by eliminating protocol conversion.
[0024] See now Figure 2The diagram illustrates an apparatus 200 for transmission across NoC tunnels. This apparatus includes a flow controller 202 configured to encapsulate NBSP flits into UCIe flit format. The flow controller uses a PTF as a token to encapsulate the NBSP flit format into UCIe flit format. The PTF is appended to the header of the NBSP flit as a message identifier and carries the message length within the message token to frame the message end boundary. Additionally, the PTF also carries the message type, virtual channel and physical channel, and local NBSP size for transmission to the remote die.
[0025] When NBSP messages are framed using PTF, the message is arranged starting with the first (1st) data word (DW) of the UCIe flit format payload. The next NBSP flit can be appended directly to the NBSP message being transmitted, and this process can continue as needed. When constructing NBSP flit formats using PTF, there are no UCIe flit boundary constraints, allowing NBSP flits to extend across multiple UCIe flits as needed. In another embodiment, the PTF format can be used on all UCIe flit formats, particularly flit format 6, which maximizes UCIe bandwidth performance.
[0026] Table 1 shows the PTF payload sizes. Tokens can support message payload sizes ranging from a minimum of 1 DW to a maximum of 1024 DW. In a higher performance efficiency mode, a single PTF can be combined with multiple NBSP flit formats to improve performance.
[0027]
[0028] Table 1 Table 2 shows the PTF of the flow controller, and Table 3 shows the PTF definition of the flow controller. This invention supports up to 16 physical channels and 16 virtual channels, enabling direct encapsulation of NBSP physical and virtual channels into UCIe links. The PTF can tunnel any type of NBSP message, including consistent and inconsistent data streams.
[0029]
[0030] Table 2
[0031] Table 3 According to embodiments of the present invention, each channel transmission of address writing, address reading, and write response corresponds to a single message. Furthermore, for write or read channels, the complete payload is transmitted as a single message in a single transmission, or the complete payload is combined and transmitted multiple times as a single message.
[0032] PTF is not limited to NBSP message formatting; it also supports AMBA AXI and other protocols. For the AMBAAXI protocol, each transmission on the AW, AR, and BRSP channels corresponds to a single message with a single token identifier assigned to its header. For W and R channels, the payload can be sent as a single message in a single transmission (a complete payload of the transaction size, measured in data width), or, in performance mode, it can be combined into multiple transmissions and sent as a single message to reduce token overhead.
[0033] In the following text, it will be based on Figures 3 to 7 Embodiments of the invention are provided for a more detailed explanation. Through these examples, the advantages of the invention can be more readily understood and put into practice. However, it should be understood that the following examples do not limit the scope of protection of the invention in any way.
[0034] Example Example 1: Encapsulate NBSP or AXI into two UCIe flit formats. Figure 3 An example of NBSP or AXI encapsulation in 2UCIe flit format 4 is shown.
[0035] Example 2: Encapsulate NBSP or AXI into two UCIe flit formats. Figure 4 An example of NBSP or AXI encapsulation in 2UCIe flit format 6 is shown.
[0036] Example 3: Encapsulation of the NBSP protocol Figure 5 and Figure 6 The NBSP protocol encapsulated in the payload is shown. Figure 5 The NBSP with payload is shown, while Figure 6 The NBSP without a payload is shown.
[0037] Example 4: An NBSP message with a payload is followed by three NBSP messages without payloads and is encapsulated into a 256-bit wide FDI interface before flit formatting.
[0038] Table 4 shows examples of NBSP messages encapsulated in PTF format with and without payload, as well as three examples of NBSP messages encapsulated in PTF format without payload. Here, Token represents a token, and Flit represents a flit.
[0039]
[0040] Table 4 Example 5: An NBSP message with a payload is followed by three NBSP messages without payloads, and is encapsulated in UCIe flit format 4, which is 256 bits wide for the FDI interface.
[0041] Table 5 shows various NBSP messages encapsulated in PTF format and arranged in UCIe flit format 4. Here, Token represents a token, and Flit represents a flit.
[0042]
[0043] Table 5 Example 6: Encapsulation of the AMBA AXI-4 protocol Figure 7 The AMBA AXI-4 protocol encapsulation of the payload is summarized, and examples of various parameters are listed in the table. The total number of bits for each channel is rounded up to the data word (DW) granularity.
[0044] Example 7: The W data packet is followed by the AW packet, then by the R packet, and encapsulated into a 256-bit wide FDI interface before flit formatting. Table 6 shows that before conversion to the flit format, W data packets (W Packet / W Packet Data), followed by AW packets (AW Packet / (AW Packet Data), and then R data packets (AR Packet / R Packet Data) are encapsulated using PTF with a 256-bit FDI bus width. Here, Token represents the token.
[0045]
[0046] Table 6 Example 8: A W data packet follows an AW data packet, then an R data packet. After being encapsulated into Flit format 4, it is encapsulated to a width of 256 bits for the FDI interface. Table 7 shows the messages encapsulated in UCIe flit format 4. Here, Token represents the token.
[0047]
[0048] Table 7 Example 9: A W data packet is followed by an AW data packet, and then by multiple R data packets. After being encapsulated into Flit format 4, it is encapsulated into a 256-bit wide FDI interface. Table 8 shows 2W data packets being merged into a single PTF format. Here, Token represents the token.
[0049]
[0050] Table 8 In the enhanced message token framing format, one or more PTFs can be sent before the message itself. This provides advance notice to the receiver, thus improving decoding. In this framing method, the start bit (bit 11) of the PTF indicates whether a PTF exists in the next DW. When start bit 11 is "1b", it indicates that the next DW is still a PTF token. Then, the start bit 11 of the next PTF token is daisy-chained to the next PTF token, and so on, until the start bit 11 of the last PTF token indicates "0b", indicating that this is the last PTF token, and the subsequent DW is the payload of the corresponding message. Here, Token represents the token.
[0051]
[0052] Table 9 While the above examples describe the invention in one particular embodiment, they are not exhaustive. It should be understood that the invention can be implemented in various other forms without departing from its intended scope.
Claims
1. A computer implementation method (100) for NoC tunnel transmission in a network-on-chip (NOC), characterized in that, Includes the following steps: Based on the available size within the Universal Chip Interconnect High-Speed UCIe flit format, the Message Token Framing (PTF) is applied to encapsulate one or more Nembus Protocol NBSP flits or fragments thereof into the UCIe flit format (102). The PTF is appended as a message token identifier to the NBSP flit header (104). The PTF is used to frame the NBSP flit to identify the start and end boundaries, forming an encapsulated message (106). The message length, message type, channel type, and protocol data width are carried in the PTF (108); and The encapsulated message is used for tunneling across NoC in UCIe flit format (110).
2. The method (100) according to claim 1, wherein, Each channel transmission, including address writing, address reading, and write response, corresponds to a single message.
3. The method (100) according to claim 1, wherein, For write or read channels, the complete payload is sent as a single message in a single transmission.
4. The method (100) according to claim 1, wherein, For write or read channels, the complete payload is combined and sent as a single message in multiple transmissions.
5. The method (100) according to claim 1, wherein, The 16 physical channels and 16 virtual channels of the NBSP flit are encapsulated on the UCIe flit format.
6. The method (100) according to claim 1, wherein, Cross-NOC tunneling via the UCIe flit format enables the support of both consistent and inconsistent data streams for NBSP messages.
7. An apparatus (200) for NoC tunnel transmission in a network-on-chip, comprising: The flow controller (202) is configured to encapsulate the Nembus protocol NBSP flit into the Universal Chip Interconnect High-Speed UCIeflit format, including the following steps: Based on the available size within the UCIe flit format, apply Message Token Framing (PTF) to encapsulate one or more Nembus Protocol NBSP flits or fragments thereof into a Universal Chip Interconnect High-Speed UCIe flit format. The PTF is appended as a message token identifier to the NBSP flit header; The PTF is used to frame the NBSP flit to identify the start and end boundaries, forming an encapsulated message; The message length, message type, channel type, and protocol data width are carried in the PTF; and The encapsulated message is used for cross-NoC tunnel transmission in UCIe flit format.
8. The device (200) according to claim 7, wherein, The message token frame can carry a message payload size ranging from 1 data word to 1024 data words.
Citation Information
Patent Citations
Network-On-Chip Environment and Method for Reduction of Latency
US20080186998A1
Noc routing in a multi-chip device
US20240211422A1
