Flow control initialization method and apparatus
By introducing the FC DLLP message format in the Flit-Mode of the PCIe 6.X protocol and optimizing the allocation of the Header Credit Buffer, the problem of resource waste in the PCIe bus is solved, and more efficient data transmission and resource utilization are achieved.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- WUXI STARS MICRO SYSTEM TECHNOLOGIES CO LTD
- Filing Date
- 2025-12-22
- Publication Date
- 2026-07-02
AI Technical Summary
The lack of an effective flow control mechanism during data transmission in the PCIe bus affects bandwidth performance, especially in the PCIe 6.X protocol, where Header Credit Buffer resources are wasted and cannot be flexibly allocated according to actual needs.
In the Flit-Mode mode of the PCIe 6.X protocol, a new flow control message format, FC DLLP, is introduced. By negotiating the state, the reserved size of each header field of the TLP packet is obtained, the allocation of the Header Credit Buffer is optimized, and the resources of both parties in the link are reasonably adjusted.
By optimizing the negotiation process, redundant space in the Header Credit Buffer was reduced, resource utilization efficiency was improved, and 21% to 31% of resources were saved, resulting in more efficient data transmission.
Smart Images
Figure CN2025144391_02072026_PF_FP_ABST
Abstract
Description
Flow control initialization method and device
[0001] Cross-reference of related applications
[0002] This application claims priority to Chinese patent application No. 202411965341.2, filed on December 28, 2024, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application belongs to the field of chip technology, and specifically relates to an initialization method and apparatus for flow control. Background Technology
[0004] As modern computer systems increasingly demand higher performance and lower power consumption, PCIe, as a high-bandwidth, low-latency interface standard, is widely used in various computer systems.
[0005] In earlier versions of PCIe, there was no flow control mechanism. The sending end did not know whether the receiving end could receive the corresponding TLP (Transaction Layer Packets) at any given moment. Therefore, the sending end could only attempt to send, which might be interrupted by multiple waiting cycles (due to reasons such as the receiving device not being ready), or even retries. These problems, along with the evolution of the PCIe bus and the increasing demands for data rates and transmission bandwidth, have severely impacted the bus's bandwidth performance. Summary of the Invention
[0006] According to a first aspect of this application, a flow control initialization method is provided, the method being applied to the Flit-Mode mode at the local end of a PCIe data link, comprising:
[0007] In response to entering the initialization phase of the data link layer, the system switches to the negotiation state.
[0008] In the negotiation state, obtain the reserved size corresponding to each header field of the TLP data packet;
[0009] Based on the reserved size corresponding to each header field, a first negotiation data packet is generated;
[0010] The first negotiation data packet is sent to the PCIe data link peer so that the PCIe data link peer determines the initial credit amount based on the reserved size of each header field carried in the first negotiation data packet.
[0011] In an optional implementation, during the negotiation state, obtaining the reserved size corresponding to each header field of the TLP data packet includes:
[0012] Based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type, determine the first reserved size of the basic field of the header field of the TLP packet;
[0013] Determine the second reserved size of the OHC field in the header of the TLP packet based on whether the TLP packet supports the OHC-E type;
[0014] Based on whether the header and trailer fields of the TLP packet support ECRC and IDE MAC, determine the third reserved size of the header and trailer fields of the TLP packet.
[0015] In an optional implementation, the method further includes:
[0016] Receive a second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link;
[0017] Based on the reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet, the initial credit quantity of the PCIE data link local end is determined, and the initialization state of the initialization phase of the data link layer is switched.
[0018] In the initialization state, based on the initial credit amount, flow control DLLPs corresponding to different types of TLP data packets are sent to the PCIE data link peer.
[0019] In an optional implementation, the reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet includes: a first reserved size for the basic header field, a second reserved size for the OHC header field, and a third reserved size for the tail header field; determining the initial credit quantity at the PCIE data link local end based on the reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet includes:
[0020] The header size of the TLP data packet at the peer of the PCIE data link is calculated based on the first reserved size of the basic field of the header field, the second reserved size of the OHC field of the header field, and the third reserved size of the tail field of the header field of the TLP data packet at the peer of the PCIE data link.
[0021] The initial credit count is determined based on the header size and other field sizes of the TLP data packets at the peer end of the PCIE data link, as well as the receive data buffer size at the local end of the PCIE data link.
[0022] In an optional implementation, the method further includes:
[0023] In the negotiation state, if a flow control DLLP is received from the PCIE data link peer in the initialization state of the data link layer initialization phase, then the system directly switches to the initialization state of the data link layer initialization phase.
[0024] In the initialization state, flow control DLLPs corresponding to different types of TLP data packets are sent to the PCIE data link peer based on the default credit amount.
[0025] According to a second aspect of this application, a flow control initialization apparatus is provided, the apparatus being applied in Flit-Mode mode at the local end of a PCIe data link, comprising:
[0026] The response unit is used to switch to the negotiation state in response to the initialization phase of entering the data link layer;
[0027] The acquisition unit is used to acquire the reserved size corresponding to each header field of the TLP data packet in the negotiation state;
[0028] The generation unit is used to generate a first negotiation data packet based on the reserved size corresponding to each header field;
[0029] The first sending unit is configured to send the first negotiation data packet to the PCIe data link peer, so that the PCIe data link peer determines the initial credit quantity of the PCIe data link peer based on the reserved size of each header field carried in the first negotiation data packet.
[0030] In an optional implementation, the acquisition unit includes:
[0031] The first determining subunit is used to determine the first reserved size of the basic field of the header field of the TLP packet based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type;
[0032] The second determining subunit is used to determine the second reserved size of the OHC field in the header of the TLP data packet based on whether the TLP data packet supports the OHC-E type.
[0033] The third determining subunit is used to determine the third reserved size of the header and tail fields of the TLP data packet based on whether the header and tail fields of the TLP data packet support ECRC and IDE MAC.
[0034] In an optional embodiment, the apparatus further includes:
[0035] The receiving unit is configured to receive a second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link;
[0036] The determining unit is used to determine the initial credit quantity of the PCIE data link local end based on the reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet, and switch to the initialization state of the initialization phase of the data link layer.
[0037] The second sending unit is used to send flow control DLLPs corresponding to different types of TLP data packets to the PCIE data link peer based on the initial credit amount in the initialization state.
[0038] In an optional implementation, the reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet includes: a first reserved size for the header base field, a second reserved size for the header OHC field, and a third reserved size for the header tail field; the determining unit includes:
[0039] The calculation subunit is used to calculate the header size of the TLP data packet at the peer end of the PCIE data link based on the first reserved size of the basic field of the header field, the second reserved size of the OHC field of the header field, and the third reserved size of the tail field of the header field of the TLP data packet at the peer end of the PCIE data link.
[0040] The fourth determining subunit is used to determine the initial credit quantity based on the header field size and other field sizes of the TLP data packet at the peer end of the PCIE data link and the receive data buffer size at the local end of the PCIE data link.
[0041] In an optional embodiment, the apparatus further includes:
[0042] The switching unit is configured to, in the negotiation state, if it receives a flow control DLLP sent by the PCIE data link peer in the initialization state of the data link layer initialization phase, directly switch to the initialization state of the data link layer initialization phase.
[0043] The third sending unit is used to send flow control DLLPs corresponding to different types of TLP data packets to the PCIE data link peer based on the default credit amount in the initialization state.
[0044] Details of one or more embodiments of this application are provided in the following description and accompanying drawings. Other features, objects, and advantages of this application will also become apparent from the specification, drawings, and claims. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 is a schematic diagram of the basic principle of flow control in a PCIE system based on relevant technologies.
[0047] Figure 2 is a schematic diagram of the flow control cache classification of PCIe systems based on relevant technologies.
[0048] Figure 3 is a schematic diagram illustrating the newly added reserved 5 / 6 / 7DW in the TLP Header Base of the PCIE 6.X protocol according to relevant technologies.
[0049] Figure 4 is a flowchart illustrating the initialization method for flow control according to an exemplary embodiment of this application.
[0050] Figure 5 is a schematic diagram illustrating the optimization of data link state transition according to an exemplary embodiment of this application.
[0051] Figure 6 is a structural block diagram of a flow control initialization device according to an exemplary embodiment of the present application. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0053] To ensure that the data sent by the PCIe device does not exceed the receiving capacity of the receiving device, and to avoid triggering data retransmission which would reduce data output efficiency or data overflow which would lead to data loss, the PCIe protocol provides a flow control (FC) scheme to improve the data transmission efficiency of the PCIe bus.
[0054] PCIe bus flow control uses a credit-based principle, coordinated by the transaction layer and data link layer. As shown in Figure 1, flow control DLLPs (Data Link Layer Packets) are sent between adjacent ports via the data link layer. During initialization, the receiver notifies the sender of the size of its buffer. During normal operation, the receiver also periodically informs the sender of the size of each buffer at the receiver via flow control DLLPs. Before sending, the sender can use the flow control mechanism to determine whether the receiver can receive the upcoming transaction layer packets (TLPs).
[0055] PCIe devices transmit and receive TLPs of three types: P (Posted Request) messages, NP (Non-Posted Request) messages, and CPL (Completion) messages. The receiving device's receive buffer is also divided into three categories based on these three TLP types, to store P / NP / CPLTLP messages respectively, as shown in Figure 2. Each type of buffer within the TLP is further divided into Header Buffer and Data Buffer depending on whether it stores the message header field or the message data field. The buffer storing the header field of a P type message is defined as PH (Posted Request Header buffer), and the buffer storing the data field of a P type message is defined as PD (Posted Request Data buffer). Messages storing both the header and data fields of NP / CPL types are defined as NPH / NPD and CPLH / CPLD, respectively.
[0056] The receive buffer for all types of messages on a PCIe receiving device is measured in credits, with the number of credits reflecting the size of the buffer.
[0057] In the PCIe 6.X protocol, a new transmission mode called Flit mode was added, replacing the traditional transmission mode as non-Flit mode. In traditional mode, TLPs are of variable length, requiring start and end flags at the physical layer to mark the header and trailer of data packets. Furthermore, TLPs and DLLPs operate at different layers and are independent of each other. Flit mode packages the information from multiple TLPs and one DLLP into a large data frame called a Flit. Data transmission is based on Flits, with each Flit being a fixed 256 bytes. A new TLP format is defined in this mode, with the following structure:
[0058] 1) Local TLP prefixes (optional).
[0059] 2)TLP Header Base (1DW(nop), 3DW, 4DW, 5DW(R), 6DW(R), 7DW(R)).
[0060] 3) Orthogonal Header Content (OHC) (optional), 0 to 7 DW, with a total of four types: OHC-A, OHC-B, OHC-C, and OHC-E; the specific type and size are indicated by the OHC[4:0] of the first DW in the TLP Header Base.
[0061] 4) TLP data payload.
[0062] 5) TLP Trailer (optional), the specific type and size are indicated by the TS[2:0] of the first DW in the TLP Header Base.
[0063] The portion of the TLP Header that occupies TLP Header Credit resources includes: TLP Header Base, OHC, and TLP Trailer.
[0064] According to the PCIe 6.X protocol, the three parts that are considered as message header fields are explained as follows:
[0065] 1. For TLP Header Base, four new formats have been added in FM (Flit Mode): 1DW, 5DW, 6DW, and 7DW. Among them, 1DW is NOP (empty). The other three lengths of message headers, 5DW, 6DW, and 7DW, have 40 encoding types out of the 256 encodings of 8-bit type in FM mode. However, in the current protocol, all of them are marked as Reserved. Some protocol screenshots are shown in Figure 3.
[0066] 2. Regarding OHCs, the main focus here is on the size of various OHC types. The PCIe 6.X protocol provides four main categories of OHCs: OHC-A, OHC-B, OHC-C, and OHC-E, with their sizes shown in Table 1 below:
[0067] Table 1
[0068] Among them, OHC-A / B / C may coexist, and OHC-E is equivalent to the E-ETLP Prefix in the PCIE 6.X and earlier protocols, and is an optional part;
[0069] 3. For PCIe Trailers, the specific type and size are indicated by the TS[2:0] of the first DW in the TLP Header Base. The Trailer size and meaning corresponding to the TS values are shown in Table 2 below:
[0070] Table 2
[0071] For PCIe Trailers, similar to the TLP Digiest in the PCIE 6.X and earlier protocols, and enhanced in the latest protocol, but still optional content, with sizes ranging from 0 to 5 DW.
[0072] According to the PCIe 6.X protocol, the calculation of the Header credit unit size (header field size) corresponding to the TLP Header under FM (Flit-Mode) is shown in Table 3 below:
[0073] Table 3
[0074] For both Switch (switch port receiver) and EP / RP (endpoint upstream port / root port receiver), the calculation rules for FC (Flow control) Header credit unit size (i.e., TLP header field size) are consistent. Referring to the analysis of the size of each component above, the size of one Header credit unit size can be calculated as follows:
[0075] Header Credit Unit size = 1 largest Base hdr (TLP Header Base) (7DW) + OHC-A (1DW) + OHC-B (1DW) + OHC-C (1DW) + OHC-E (4DW) + 1 largest TLP Trailer (5DW) = 7 + 1 + 1 + 1 + 4 + 5 = 19DW.
[0076] In the PCIe 6.X protocol, the need for a higher number of Header Credits increases exponentially with the increased speed. Simultaneously, the size of a single Header Credit Unit (DW) has more than doubled to 19 (compared to a maximum of 9 DW for PCIe 5.0 and earlier protocols, excluding the Local-prefix). In FM mode, this expansion of TLP and its reservation for future integration of more features into the protocol also places higher demands on the device's storage resources.
[0077] Based on the preceding analysis of the components of the TLP Header base, we can see that:
[0078] 1. The largest TLP header base size is 7DW, but 5DW, 6DW, and 7DW TLPs are currently reserved types.
[0079] 2. A total of 7DW is reserved for various OHC types, but for optional OHC-E type messages, if OHC-E is not supported, then OHC only needs to occupy 3DW space.
[0080] 3. For TLP Trailers, if only ECRC is retained, only 1DW is needed, and in most cases, such a large space as 5DW is not required.
[0081] Therefore, the required Header Credit Unit size is calculated according to the different supported characteristics as shown in Table 4 below:
[0082] Table 4
[0083] In Table 4, in the first row, when TLP base hdr5 / 6 / 7 is not supported, the maximum space occupied by TLP base hdr is 3DW. When OHC-E is not supported, the maximum space occupied by OHC is 4DW. Since TLP Trailer does not support ECRC or IDE, its space is 0. Therefore, in the first row, the size of the Header Credit Unit can be 3+4=7. The second to fourth rows support different fields respectively; therefore, the space occupied by the supported fields should be added to the base size.
[0084] In the PCIe 6.X protocol, the protocol recommends reserving Header credit buffer resources for a Header Credit Unit under the worst-case scenario. However, according to the calculations above, it is difficult to fully utilize these resources in practice. Even if we exclude the cases where the current protocol still reserves resources, the size of a Header Credit Unit can be reduced to 15 DW. Compared to the recommended value of 19 DW, there is still a 21% resource optimization space.
[0085] Because the protocol recommends reserving resources in the worst-case scenario, its Flow Control initialization phase lacks a step to negotiate the Header Credit Size; it only negotiates the Credit quantity. This could lead to a serious waste of the Header Credit buffer.
[0086] Based on the existing PCIe protocol, this application proposes a new flow control message format (FC DLLP) in Flit-Mode. Utilizing this message format, it proposes an optimized PCIe initialization flow control process. By executing this process, the latest devices supporting the PCIe 6.X protocol do not need to reserve Header Credit Buffer resources according to the worst-case scenario. Both ends of the PCIe link can accurately determine the characteristics supported by the other end that may affect the Header Credit Buffer size through the new flow control negotiation process. Based on the negotiation results, the local end can flexibly allocate and calculate the local credit amount, achieving optimized adjustment of Header Credit Buffer resources on both sides.
[0087] Referring to Figure 4, this application exemplarily proposes a flow control initialization method, which is applied to the Flit-Mode mode at the local end of the PCIe data link, including:
[0088] In step S401, in response to entering the initialization phase of the data link layer, the system switches to the negotiation state;
[0089] In step S402, under negotiation state, the reserved size corresponding to each header field of the TLP data packet is obtained;
[0090] In step S403, a first negotiation data packet is generated based on the reserved size corresponding to each header field;
[0091] In step S404, the first negotiation data packet is sent to the PCIe data link peer so that the PCIe data link peer determines the initial credit quantity of the PCIe data link peer based on the reserved size of each header field carried in the first negotiation data packet.
[0092] For example, after the PCIe 6.X protocol introduced the FM (Flit-Mode) mode, the number of DLLP message types expanded from the original 46 to 181, still using 8-bit DLLP type encoding. To facilitate the confirmation of the Header credit unit size (i.e., the header field size), a new DLLP message type was added for both parties in the link to negotiate characteristics affecting the Header credit unit size.
[0093] From the reserved code type of DLLP Type encoding, select 0011 1V2V1V0 (V2V1V0 represents the virtual channel VC number) as the new custom FC DLLP message type initFC_hdr_size. The message format definition is shown in Table 5 below:
[0094] Table 5
[0095] In this context, 1) Byte 0 still represents the DLLP type; 2) Bytes 1, 2, and 3 represent the worst-case DW size in the Header base, OCH-E (OCH-E type in the OHC field), and TLP Trailer of the P / NP / CPL TLP message, respectively, with the same format. The specific meanings of the new FC DLLP (also referred to as the negotiation data packet in this application for ease of description; the first negotiation data packet corresponds to the local end of the PCIE data link, and the second negotiation data packet corresponds to the peer end of the PCIE data link) field segment and its corresponding reserved size are shown in Table 6 below:
[0096] Table 6
[0097] In this context, bits [7:5] represent the number of Data Written Text (DW) reserved for the Header base; bits [4:3] represent the number of DWs reserved for the OHC-E; and bits [2:0] represent the number of DWs reserved for the TLP Trailer. The 16-bit CRC checksum included in the original FC DLLP message is no longer generated separately in Flit Mode, but is instead included in the Flit TLP for unified calculation.
[0098] In the Flit-Mode mode of the PCIe 6.X protocol, the transmission of DLLP during the DL_INIT phase (data link layer initialization phase) has been optimized and improved.
[0099] For example, taking the PCIe data link itself as an example, the optimization of the DL (Data Link) state machine transition as shown in Figure 5 is explained in detail below:
[0100] 1) A new state, FC_INIT_HDR_SIZE (referred to as the negotiation state for ease of description), is added. After the PCIe data link local end enters the DL_INIT phase (data link layer initialization phase), it first enters this negotiation state and sends a newly added FC DLLP initFC_hdr_size (first negotiation data packet, which carries the reserved size corresponding to each header field of the TLP data packet) to the PCIe data link peer to negotiate the size of the TLP header credit unit (TLP data packet header field). Specifically, the reserved size corresponding to each header field of the TLP data packet is sent to the PCIe data link peer through the first negotiation data packet, specifying the maximum number of DWs (Data Warehouses) occupied by each header field supported by the device, such as Header base, OHC-E, and TLP_Trailer. It should be noted that the local end of the PCIe data link can be either a data receiver or a data sender. If the local end of the PCIe data link is a data receiver, then the remote end of the PCIe data link is a data sender; conversely, if the local end of the PCIe data link is a data sender, then the remote end of the PCIe data link is a data receiver.
[0101] 2) After receiving the first negotiation data packet, the peer end of the PCIe data link calculates the actual required Header credit unit size, i.e., the size of the entire header field of the TLP data packet, based on the information carried in the first negotiation data packet. Then, based on the resources of its own Header credit receive buffer, it calculates the maximum number of Header credits it can support, i.e., the initial credit amount. It is understandable that during data transmission, as data is received and stored in the receive data buffer, this initial credit amount will change, and this change will be periodically sent to the peer end in subsequent processes.
[0102] In some optional implementations, during the negotiation phase, the reserved size corresponding to each header field of the TLP packet is obtained, including:
[0103] Based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type, determine the first reserved size of the basic field of the header field of the TLP packet;
[0104] Determine the second reserved size of the OHC field in the header of the TLP packet based on whether the TLP packet supports the OHC-E type;
[0105] Based on whether the header and trailer fields of the TLP packet support ECRC and IDE MAC, determine the third reserved size of the header and trailer fields of the TLP packet.
[0106] For example, as described above, the header size of a TLP packet depends on three fields: the Header base field, the OHC field, and the TLP Trailer field. The maximum size of these three fields is as described above. The Header base field depends on whether the new message types added in Flit-Mode are reserved. That is, if all four new formats added in Flit-Mode (1DW, 5DW, 6DW, and 7DW) are marked as Reserved, then the Header base field only needs to retain the original part, meaning the maximum size is 4DW. However, if not all four new formats added in Flit-Mode (1DW, 5DW, 6DW, and 7DW) are marked as Reserved, the maximum size of the Header base field can be determined as 5DW, 6DW, or 7DW depending on the actual situation.
[0107] Similarly, for the OHC header field, in Flit-mode, a total of 7DW is reserved for various OHC types. However, if OHC-E is not supported, OHC only needs to occupy 3DW of space. Therefore, the maximum size of the OHC header field (3DW or 7DW) can be determined based on whether OHC-E is supported. For the footer fields in the header field, if only ECRC is retained, only 1DW is needed. If IDE MAC (Integrated Development Environment) is also supported, then 4DW is required. Therefore, the reserved size of the footer fields in the header field can be determined by whether ECRC and IDE MAC are supported.
[0108] In some alternative implementations, the method also includes:
[0109] Receive the second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link;
[0110] Based on the reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet, the initial credit quantity at the PCIE data link local end is determined, and the system switches to the initialization state of the data link layer initialization phase.
[0111] In the initialization state, based on the initial credit amount, different types of TLP data packets corresponding to flow control DLLPs are sent to the PCIE data link peer.
[0112] For example, after the PCIe data link local end enters the data link layer initialization phase, it first switches to the negotiation state and sends a first negotiation data packet to the PCIe data link peer end. At the same time, the PCIe data link peer end, like the PCIe data link local end, also first switches to the negotiation state after entering the data link layer initialization phase and sends a second negotiation data packet to the PCIe data link local end. The first negotiation data packet and the second negotiation data packet are the same, except that the second negotiation data packet is sent from the PCIe data link peer end to the PCIe data link local end, and the second negotiation data packet carries the reserved size of each header field of the PCIe data link peer end.
[0113] For example, if the peer of the PCIe data link sends a second negotiation data packet to the local PCIe data link and it is received, the local PCIe data link will calculate the space occupied by the header fields of the peer's TLP data packet, i.e., the header field size, based on the reserved size of each header field carried in the second negotiation data packet. This allows the local PCIe data link to determine the maximum space the header fields of the TLP data packet received from the peer will occupy. Since the space occupied by the data payload (excluding the header fields) in the TLP data packet is fixed, the maximum space occupied by a TLP data packet can be determined based on the maximum header field size and the data payload size. Therefore, given the known size of the received data buffer, the local PCIe data link can determine its initial credit limit. This credit limit is provided by the local PCIe data link to the peer, and when the peer sends data to the local PCIe data link, flow control measures are taken based on this credit limit.
[0114] For example, after receiving the second negotiation data packet sent by the PCIe data link peer and completing the calculation of the initial credit amount, the PCIe data link local end enters the initialization state FC_INIT1 of the data link layer initialization phase. The initialization state FC_INIT1 is a state specified by the protocol. In this state, the PCIe data link local end sends an initFC-P / NP / CPL DLLP message to the PCIe data link peer, that is, sends the flow control DLLP corresponding to different types of TLP data packets to the PCIe data link peer, so as to inform the PCIe data link peer of the initial credit amount corresponding to different types of TLP data packets.
[0115] In some optional implementations, the reserved size of each header field of the TLP packet at the PCIe data link peer in the second negotiation packet includes: a first reserved size for the basic header field, a second reserved size for the OHC header field, and a third reserved size for the tail header field; based on the reserved size of each header field of the TLP packet at the PCIe data link peer in the second negotiation packet, the initial credit quantity at the PCIe data link local end is determined, including:
[0116] Calculate the header size of the TLP data packet at the peer end of the PCIE data link based on the first reserved size of the basic header field, the second reserved size of the OHC header field, and the third reserved size of the tail header field of the TLP data packet at the peer end of the PCIE data link.
[0117] The initial credit count is determined based on the header field size and other field sizes of the TLP data packets at the peer end of the PCIe data link, as well as the receive data buffer size at the local end of the PCIe data link.
[0118] For example, the PCIe data link peer, like the PCIe data link local end, sends a second negotiation data packet including a first reserved size for the header base field, a second reserved size for the header OHC field, and a third reserved size for the header tail field. The PCIe data link local end calculates the header field size of the PCIe data link peer's TLP data packet based on these three reserved sizes in the second negotiation data packet. The PCIe data link peer's TLP data packet is received by the PCIe data link local end and stored in the receive data buffer; therefore, the initial credit amount needs to be calculated based on the header field size of the PCIe data link peer's TLP data packet, the sizes of other fields in the TLP data packet, and the receive data buffer size. The sizes of other fields in the TLP data packet may include, for example, the size of the effective data payload, which is typically fixed.
[0119] In some alternative implementations, the method also includes:
[0120] In the negotiation state, if a flow control DLLP is received from the PCIe data link peer in the initialization state of the data link layer initialization phase, then the system directly switches to the initialization state of the data link layer initialization phase.
[0121] In the initialization state, based on the default credit amount, different types of TLP data packets corresponding to flow control DLLPs are sent to the PCIe data link peer.
[0122] For example, as described above, in order to flexibly allocate and calculate the local credit quantity and optimize the data buffer resources received between the local and peer ends, this application adds a new state, namely the negotiation state, after entering the initial stage of the data link layer. In this state, the local end sends a first negotiation data packet to the peer end, and the peer end also sends a second negotiation data packet to the local end. However, if the peer end is still the original, unmodified device and does not support the scheme provided by this application, the PCIe data link local end will not receive the second negotiation data packet sent by the peer end in the negotiation state. Instead, it will directly receive the flow control DLLP of various TLP data packets sent by the PCIe data link peer end in the initialization state of the data link layer initialization phase. In this case, the PCIe data link local end can determine that the peer end is the original, unmodified device and does not support the scheme provided by this application. Therefore, it needs to follow the protocol's provisions and no longer negotiate the header field size of the TLP data packet, nor recalculate the initial credit quantity, but directly use the default credit quantity. The default credit quantity is calculated based on the maximum occupied size of 19DW for the header field size of the TLP data packet. Therefore, after the local end of the PCIe data link directly receives the flow control DLLPs for various TLP data packets sent by the peer in the initialization state of the data link layer initialization phase, it switches to the initialization state of the data link layer initialization phase and also sends flow control DLLPs for various TLP data packets to the peer. The credit counts in these flow control DLLPs are all determined based on the default credit count. It should be noted that after entering FC_INIT1, the process of both the peer and the local end of the PCIe data link is consistent with the original PCIe 6.X initialization process until entering the DL_Active state.
[0123] Based on the previous analysis, for a PCIe 6.X-enabled device, assuming the Header Credit buffer reserves 128 credits in the worst-case scenario as required by the protocol, the change in the number of credits after negotiation and adjustment through the newly added FC_INIT_HDR_SIZE process is shown in Table 7 below:
[0124] Table 7
[0125] The calculations in the table show that with the addition of the FCINIT_HDR_SIZE state and related negotiation process, PCIe 6.X devices no longer need to reserve resources based on the worst-case scenario. Instead, the number of Header Credits is accurately calculated based on the size of the Header Credit unit after the two parties in the PCIe link have negotiated and synchronized, combined with the size of the local Header Credit buffer resources. This can save redundant cache space and increase the number of Header Credits by 126% to 271%, enabling a more reasonable allocation of resources.
[0126] Accordingly, referring to Figure 6, this application exemplarily proposes a flow control initialization device applied in the Flit-Mode mode at the local end of the PCIe data link, including:
[0127] Response unit 601 is used to switch to negotiation state in response to entering the initialization phase of the data link layer;
[0128] The acquisition unit 602 is used to acquire the reserved size corresponding to each header field of the TLP data packet in the negotiation state;
[0129] The generation unit 603 is used to generate a first negotiation data packet based on the reserved size corresponding to each header field.
[0130] The first sending unit 604 is used to send the first negotiation data packet to the PCIe data link peer, so that the PCIe data link peer determines the initial credit quantity of the PCIe data link peer according to the reserved size of each header field carried in the first negotiation data packet.
[0131] In some alternative implementations, the acquisition unit includes:
[0132] The first determining subunit is used to determine the first reserved size of the basic field of the header field of the TLP packet based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type;
[0133] The second determining subunit is used to determine the second reserved size of the OHC field in the header of the TLP data packet based on whether the TLP data packet supports the OHC-E type.
[0134] The third determining subunit is used to determine the third reserved size of the header and tail fields of the TLP data packet based on whether the header and tail fields of the TLP data packet support ECRC and IDE MAC.
[0135] In some alternative implementations, the device also includes:
[0136] The receiving unit is used to receive the second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link.
[0137] The determining unit is used to determine the initial credit quantity of the PCIE data link local end based on the reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet, and switch to the initialization state of the initialization phase of the data link layer.
[0138] The second sending unit is used, in the initialization state, to send flow control DLLPs corresponding to different types of TLP data packets to the PCIE data link peer based on the initial credit quantity.
[0139] In some optional implementations, the reserved size of each header field of the TLP packet at the PCIe data link peer in the second negotiation packet includes: a first reserved size for the basic header field, a second reserved size for the OHC header field, and a third reserved size for the tail header field; the determining unit includes:
[0140] The calculation subunit is used to calculate the header size of the TLP data packet at the PCIE data link peer based on the first reserved size of the header field base field, the second reserved size of the header field OHC field, and the third reserved size of the header field tail field of the TLP data packet at the PCIE data link peer.
[0141] The fourth determination subunit is used to determine the initial credit quantity based on the header field size, other field size of the TLP data packet at the PCIE data link peer, and the receive data buffer size at the PCIE data link local end.
[0142] In some alternative implementations, the device also includes:
[0143] The switching unit is used to directly switch to the initialization state of the data link layer if it receives a flow control DLLP sent by the PCIe data link peer in the initialization state of the initialization phase of the data link layer during the negotiation state.
[0144] The third sending unit is used, in the initialization state, to send the flow control DLLP corresponding to different types of TLP data packets to the PCIE data link peer based on the default credit quantity.
[0145] The above-described apparatus is used to implement the flow control initialization method provided in the above embodiments. For specific details, please refer to the description of the flow control initialization method in the above embodiments, which will not be repeated here.
[0146] Compared with related technologies, the technical solutions of this application, based on different embodiments, have at least the following advantages:
[0147] This application adds the FCINIT_HDR_SIZE state and related negotiation process to the original PCIe protocol. For PCIe 6.X devices, it eliminates the need to reserve resources based on worst-case scenarios. Instead, it accurately calculates the number of Header Credits based on the size of the Header Credit unit after negotiation and synchronization between both parties in the PCIe link, combined with the size of the local Header Credit buffer resource. This saves redundant cache space and increases the number of Header Credits by 126% to 271%, enabling more rational resource allocation. Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description, or may be learned by practicing this application. The objectives and other advantages of this application can be realized and obtained through the structures and processes shown in the description and accompanying drawings.
[0148] It is understood that the circuit structures, names, and parameters described in the above embodiments are merely examples. Those skilled in the art can also make readily conceived combinations and adjustments to the structural features of the above embodiments according to their needs, and the concept of this application should not be limited to the specific details of the above examples.
[0149] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A flow control initialization method, applied to the Flit-Mode mode on the local end of a PCIe data link, comprising: In response to entering the initialization phase of the data link layer, the system switches to the negotiation state. In the negotiation state, obtain the reserved size corresponding to each header field of the TLP data packet; Based on the reserved size corresponding to each header field, a first negotiation data packet is generated; The first negotiation data packet is sent to the PCIe data link peer so that the PCIe data link peer determines the initial credit amount based on the reserved size of each header field carried in the first negotiation data packet.
2. The initialization method for flow control according to claim 1, wherein, In the negotiation state, the reserved size corresponding to each header field of the TLP data packet is obtained, including: Based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type, determine the first reserved size of the basic field of the header field of the TLP packet; Determine the second reserved size of the OHC field in the header of the TLP packet based on whether the TLP packet supports the OHC-E type; Based on whether the header and trailer fields of the TLP packet support ECRC and IDE MAC, determine the third reserved size of the header and trailer fields of the TLP packet.
3. The flow control initialization method according to claim 1 or 2 further includes: Receive a second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link; Based on the reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet, the initial credit quantity of the PCIE data link local end is determined, and the initialization state of the initialization phase of the data link layer is switched. In the initialization state, based on the initial credit amount, flow control DLLPs corresponding to different types of TLP data packets are sent to the PCIE data link peer.
4. The initialization method for flow control according to claim 3, wherein, The reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet includes: a first reserved size for the basic header field, a second reserved size for the OHC header field, and a third reserved size for the tail header field; determining the initial credit quantity at the PCIE data link local end based on the reserved size of each header field of the TLP data packet at the PCIE data link peer in the second negotiation data packet includes: The header size of the TLP data packet at the peer of the PCIE data link is calculated based on the first reserved size of the basic field of the header field, the second reserved size of the OHC field of the header field, and the third reserved size of the tail field of the header field of the TLP data packet at the peer of the PCIE data link. The initial credit count is determined based on the header size and other field sizes of the TLP data packets at the peer end of the PCIE data link, as well as the receive data buffer size at the local end of the PCIE data link.
5. The flow control initialization method according to any one of claims 1, 2, and 4 further comprises: In the negotiation state, if a flow control DLLP is received from the PCIE data link peer in the initialization state of the data link layer initialization phase, then the system directly switches to the initialization state of the data link layer initialization phase. In the initialization state, flow control DLLPs corresponding to different types of TLP data packets are sent to the PCIE data link peer based on the default credit amount.
6. A flow control initialization device, applied in Flit-Mode mode at the local end of a PCIe data link, comprising: The response unit is used to switch to the negotiation state in response to the initialization phase of entering the data link layer; The acquisition unit is used to acquire the reserved size corresponding to each header field of the TLP data packet in the negotiation state; The generation unit is used to generate a first negotiation data packet based on the reserved size corresponding to each header field; The first sending unit is configured to send the first negotiation data packet to the PCIe data link peer, so that the PCIe data link peer determines the initial credit quantity of the PCIe data link peer based on the reserved size of each header field carried in the first negotiation data packet.
7. The initialization device for flow control according to claim 6, wherein, The acquisition unit includes: The first determining subunit is used to determine the first reserved size of the basic field of the header field of the TLP packet based on whether the newly added message type of Flit-Mode in the TLP packet is a reserved type; The second determining subunit is used to determine the second reserved size of the OHC field in the header of the TLP data packet based on whether the TLP data packet supports the OHC-E type. The third determining subunit is used to determine the third reserved size of the header and tail fields of the TLP data packet based on whether the header and tail fields of the TLP data packet support ECRC and IDE MAC.
8. The flow control initialization device according to claim 6 or 7, further comprising: The receiving unit is configured to receive a second negotiation data packet sent by the peer of the PCIe data link; wherein the second negotiation data packet carries the reserved size of each header field of the TLP data packet of the peer of the PCIe data link; The determining unit is used to determine the initial credit quantity of the PCIE data link local end based on the reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet, and switch to the initialization state of the initialization phase of the data link layer. The second sending unit is used to send flow control DLLPs corresponding to different types of TLP data packets to the PCIE data link peer based on the initial credit amount in the initialization state.
9. The initialization device for flow control according to claim 8, wherein, The reserved size of each header field of the TLP data packet of the PCIE data link peer in the second negotiation data packet includes: the first reserved size of the header field base field, the second reserved size of the header field OHC field, and the third reserved size of the header field tail field; The determining unit includes: The calculation subunit is used to calculate the header size of the TLP data packet at the peer end of the PCIE data link based on the first reserved size of the basic field of the header field, the second reserved size of the OHC field of the header field, and the third reserved size of the tail field of the header field of the TLP data packet at the peer end of the PCIE data link. The fourth determining subunit is used to determine the initial credit quantity based on the header field size and other field sizes of the TLP data packet at the peer end of the PCIE data link and the receive data buffer size at the local end of the PCIE data link.
10. The flow control initialization device according to any one of claims 6, 7, and 9, further comprising: The switching unit is configured to, in the negotiation state, if it receives a flow control DLLP sent by the PCIE data link peer in the initialization state of the data link layer initialization phase, directly switch to the initialization state of the data link layer initialization phase. The third sending unit is used to send flow control DLLPs corresponding to different types of TLP data packets to the PCIE data link peer based on the default credit amount in the initialization state.