Flow control between peripheral component interconnect express devices

Abstract

Aspects of the present disclosure provide techniques for resending a transactional layer packet (TLP) that received a negative acknowledgement (NACK) without resending a previously transmitted TLP that is received correctly but out of order by a receiver. A receiver (e.g., a receive link partner) may provide a NACK including a start sequence number (SSN) and an end sequence number (ESN) to a transmitter (e.g., a transmit link partner), the NACK may inform the transmitter of packets for retransmission and / or packets that may be cleared from a transmit buffer of the transmitter.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to pending U.S. non - provisional application Ser. No. 17 / 976,468, filed Oct. 28, 2022, which is assigned to the assignee of this application and is hereby incorporated by reference in its entirety as if set forth fully herein and for all applicable purposes. Technical Field

[0003] The techniques discussed below generally relate to Peripheral Component Interconnect Express (PCIe) devices and, more particularly, to techniques for flow control in communication between PCIe devices. Background Art

[0004] High - speed interfaces are often used between the circuits and components of mobile wireless devices and other complex systems. For example, certain devices may include processing devices, communication devices, storage devices, and / or display devices that interact with each other via one or more high - speed interfaces. Some of these devices (including Synchronous Dynamic Random Access Memory (SDRAM)) may be capable of providing or using data and control information at the processor clock rate. Other devices (e.g., display controllers) may use variable amounts of data at a relatively low video refresh rate.

[0005] The Peripheral Component Interconnect Express (PCIe) standard is a high - speed interface that supports high - speed data links capable of sending data at gigabits per second. The PCIe interface also has multiple standby modes for when the link is inactive. Compared to parallel buses, PCIe can provide lower latency and higher data transfer rates. PCIe can be used for communication between a wide variety of different devices. Typically, one device (e.g., a processor or a hub) acts as a host that communicates with multiple devices (called endpoints) via a PCIe link (data link). Peripheral devices or components can include graphics adapter cards, network interface cards (NICs), storage accelerator devices, mass storage devices, input / output (I / O) interfaces, and other high - performance peripheral devices.

[0006] Flow control between PCIe devices can be handled at the transaction layer that executes flow control of Transaction - Layer Packets (TLPs). PCIe communication involves the sending and receiving of TLPs between PCIe devices. The data link layer applies a sequence number to each TLP to facilitate flow control and the correct ordering of received TLPs. The receiver can re - send corrupted or lost TLPs based on the sequence number of the received TLP. Summary of the Invention

[0007] The following presents a generalization of one or more specific implementations to provide a basic understanding of such specific implementations. This generalization is not an exhaustive overview of all expected specific implementations and is not intended to identify key or important elements of all specific implementations, nor is it intended to delineate the scope of any or all specific implementations. Its sole purpose is to present some concepts of one or more specific implementations in a simplified form as a prelude to the more detailed description that follows.

[0008] In one example, a device having an interface circuit and a controller for a wired data link (e.g., a Peripheral Component Interconnect Express (PCIe) link) for connecting to a link partner is disclosed. The device includes: an interface circuit configured to provide an interface to the wired data link; and a controller. The controller is configured to send one or more first data packets to the link partner via the wired data link. The controller is further configured to receive negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets in the first data packets. The controller is further configured to retransmit the requested one or more first data packets.

[0009] In one example, a device having an interface circuit and a controller for a wired data link (e.g., a PCIe link) for connecting to a link partner is disclosed. The device includes: an interface circuit configured to provide an interface to the wired data link; and a controller. The controller is configured to receive one or more first data packets from the link partner via the wired data link, the one or more first data packets being included in an expected sequence of data packets. The controller is further configured to send negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier in time than the corresponding sequence numbers of the one or more first data packets. The controller is further configured to receive the retransmission of the one or more second data packets.

[0010] In one example, a method of operating a device for data communication is disclosed. The method includes sending one or more first data packets to a link partner via a wired data link. The method further includes receiving negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets in the first data packets. The method further includes retransmitting the requested one or more first data packets.

[0011] In one example, a method of operating a device for data communication is provided. The method includes receiving, via a wired data link, one or more first data packets from a link partner, the one or more first data packets being included in an expected sequence of data packets. The method further includes transmitting negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request a retransmission of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier in time than a corresponding sequence number of the one or more first data packets. The method further includes receiving the retransmission of the one or more second data packets.

[0012] To achieve the foregoing and related ends, one or more embodiments include the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more embodiments. However, these aspects are merely indicative of the various ways in which the principles of the embodiments may be employed and the described embodiments are intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram of a computing architecture having a Peripheral Component Interconnect Express (PCIe) interface suitable for aspects of the present disclosure.

[0014] Figure 2 is a block diagram of a system including a host system and an endpoint device system in accordance with aspects of the present disclosure.

[0015] Figure 3 is a block diagram conceptually illustrating an exemplary flow control of transaction layer packets (TLPs) between link partners in accordance with some aspects of the present disclosure.

[0016] Figure 4 is a diagram illustrating a negative acknowledgment packet (NACK) format in accordance with some aspects of the present disclosure.

[0017] Figure 5 is a flowchart illustrating a process of selectively retransmitting TLPs at a transmitter in accordance with some aspects of the present disclosure.

[0018] Figure 6 is a flowchart illustrating a process of receiving TLPs at a receiver in accordance with some aspects of the present disclosure.

[0019] Figure 7 is a diagram illustrating an example of selective TLP retransmission in accordance with some aspects of the present disclosure.

[0020] Figure 8A block diagram of a PCIe link interface processing circuit in accordance with aspects of the present disclosure.

[0021] Figure 9 A flowchart of a first exemplary method for flow control of a communication link in accordance with aspects of the present disclosure.

[0022] Figure 10 A flowchart of a second exemplary method for flow control of a communication link in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0023] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, the concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.

[0024] Aspects of the present disclosure provide techniques for retransmitting TLPs that receive negative acknowledgments (NACKs) without retransmitting previously transmitted TLPs that were received correctly but out of order by a receiver. In some aspects, a receiver (e.g., a receiving link partner) may provide a NACK to a transmitter (e.g., a transmitting link partner) that includes a start sequence number (SSN) and an end sequence number (ESN), which may notify the transmitter of packets for retransmission and / or packets that may be cleared from the transmitter's transmit buffer.

[0025] Figure 1It is a block diagram of an exemplary computing architecture using a PCIe interface. The computing architecture 100 operates using multiple high-speed PCIe interface serial links. The PCIe interface can be characterized as a device including a point-to-point topology, where individual serial links connect each device to a host, which can be referred to as the root complex 104. In the computing architecture 100, the root complex 104 couples the processor 102 to memory devices (e.g., memory subsystem 108) and PCIe switch circuitry 106. In some instances, the PCIe switch circuitry 106 includes cascaded switching devices. One or more PCIe endpoint devices 110 can be directly coupled to the root complex 104, while other PCIe endpoint devices 112-1, 112-2, ..., 112-N can be coupled to the root complex 104 through the PCIe switch circuitry 106. The root complex 104 can be coupled to the processor 102 using a proprietary local bus interface or a standard-defined local bus interface. The root complex 104 can control configuration and data transactions through the PCIe interface and can generate transaction requests for the processor 102. In some examples, the root complex 104 is implemented in the same integrated circuit (IC) device including the processor 102. The root complex 104 can support multiple PCIe ports.

[0026] The root complex 104 can control the communication between the processor 102 and the memory subsystem 108, which is an example of an endpoint. The root complex 104 (host) also controls the communication between the processor 102 and other PCIe endpoint devices 110, 112-1, 112-2, ..., 112-N. The PCIe interface can support full-duplex communication between any two endpoints, and there is no inherent limit on concurrent access across multiple endpoints. Data packets can carry information through any PCIe link. In a multi-channel PCIe link, the packet data can be striped across multiple channels. The number of channels in a multi-channel link can be negotiated during device initialization and can be different for different endpoints.

[0027] When one or both traffic directions of the channels of a PCIe link are not fully utilized by low-bandwidth applications (which can be adequately served by fewer channels), the root complex 104 and the endpoint can operate the link with more or fewer transmit and receive lines in one or both directions. In some aspects, the transmitter (e.g., the root complex 104) can use flow control techniques that can reduce the retransmission of TLPs to send TLPs to the receiver (e.g., PCIe endpoint 110).

[0028] In some aspects, the computing architecture 100 can be implemented based on the PCIe M.2 specification. The M.2 form factor can be used for mobile adapters. M.2 enables functional expansion, contraction, and higher integration into a single form factor module solution. For example, as described above regarding Figure 1Any of the described PCIe endpoints can be implemented as an M.2 adapter, and the root complex 104 can be implemented as an M.2 platform.

[0029] Figure 2 is a block diagram of an exemplary PCIe system in which aspects of the present disclosure can be implemented. The system 205 includes a host system 210 and an endpoint device system 250, which can be the same as the Figure 1 host and endpoint. For example, the host system 210 can be a PCIe M.2 platform, and the endpoint device system 250 can be an M.2 adapter. The host system 210 can be integrated on a first chip (e.g., a system-on-chip or SoC), and the endpoint device system 250 can be integrated on a second chip. Alternatively, the host system and / or the endpoint device system can be integrated in a first package and a second package (e.g., SiP), a first system board and a second system board with multiple chips, or integrated in other hardware or any combination. In this example, the host system 210 and the endpoint device system 250 are coupled by a PCIe link 285.

[0030] The host system 210 includes one or more host clients 214. Each of the one or more host clients 214 can be implemented on a processor executing software that performs the functions of the host client 214 discussed herein. For an example of more than one host client, the host clients can be implemented on the same processor or different processors. The host system 210 also includes a host controller 212, which can perform root complex functions. The host controller 212 can be implemented on a processor executing software that performs the functions of the host controller 212 discussed herein.

[0031] The host system 210 includes a PCIe interface circuit 216, a system bus interface 215, and a host system memory 240. The system bus interface 215 can interface one or more host clients 214 with the host controller 212, and interface each of the one or more host clients 214 and the host controller 212 with the PCIe interface circuit 216 and the host system memory 240. The PCIe interface circuit 216 provides an interface for the host system 210 to the PCIe link 285. In this regard, the PCIe interface circuit 216 is configured to send data (e.g., from the host client 214) to the endpoint device system 250 via the PCIe link 285 and receive data from the endpoint device system 250 via the PCIe link 285. The PCIe interface circuit 216 includes a PCIe controller 218, a physical interface 220 for a PCI Express (PIPE) interface, a physical (PHY) transmit (TX) block 222, a clock generator 224, and a PHY receive (RX) block 226. The PIPE interface 220 provides a parallel interface between the PCIe controller 218 and the PHY TX block 222 and the PHY RX block 226. The PCIe controller 218 (which can be implemented in hardware) can be configured to perform transaction layer, data link layer, and flow control functions (e.g., flow control based on the PCIe specification), as further described below. The flow control function can selectively retransmit (replay) only packets (TLPs) that receive a NACK (i.e., are lost or corrupted during transmission), rather than replaying all packets present in the transmitter's replay buffer. For example, the replay buffer can be implemented using the system memory 240 / 260 and / or be included in the PHY TX block 222 / 266.

[0032] The host system 210 also includes an oscillator (e.g., a crystal oscillator or "XO") 230 configured to generate a reference clock signal 232. In one example, the reference clock signal 232 can have a frequency of 19.2 MHz, but is not limited to this frequency. The reference clock signal 232 is input to the clock generator 224, which generates a plurality of clock signals based on the reference clock signal 232. In this regard, the clock generator 224 can include one or more phase-locked loops (PLLs), where each PLL generates a corresponding one of the plurality of clock signals by multiplying the frequency of the reference clock signal 232.

[0033] The endpoint device system 250 includes one or more device clients 254. Each device client 254 can be implemented on a processor that executes software that performs the functions of the device client 254 discussed herein. For examples of more than one device client 254, the device clients 254 can be implemented on the same processor or different processors. The endpoint device system 250 also includes a device controller 252. The device controller 252 can be configured to receive bandwidth requests from one or more device clients and determine whether to change the number of transmit lines or the number of receive lines based on the bandwidth requests. The device controller 252 can be implemented on a processor that executes software that performs the functions of the device controller.

[0034] The endpoint device system 250 includes a PCIe interface circuit 260, a system bus interface 256, and an endpoint system memory 274. The system bus interface 256 can interface one or more device clients 254 with the device controller 252 and interface each of the one or more device clients 254 and the device controller 252 with the PCIe interface circuit 260 and the endpoint system memory 274. The PCIe interface circuit 260 provides an interface for the endpoint device system 250 to the PCIe link 285. In this regard, the PCIe interface circuit 260 is configured to send data (e.g., from the device client 254) to the host system 210 (also referred to as the host device) via the PCIe link 285 and receive data from the host system 210 via the PCIe link 285. The PCIe interface circuit 260 includes a PCIe controller 262, a PIPE interface 264, a PHY TX block 266, a PHY RX block 270, and a clock generator 268. The PIPE interface 264 provides a parallel interface between the PCIe controller 262 and the PHY TX block 266 and the PHY RX block 270. The PCIe controller 262 (which can be implemented in hardware) can be configured to perform transaction layer, data link layer, and control flow functions.

[0035] The host system memory 240 and the endpoint system memory 274 at the endpoint can be configured to include registers for the status of each transmit line and receive line of the PCIe link 285. The transmit lines can be configured as differential transmit line pairs, and the receive lines can be configured as differential receive line pairs.

[0036] The endpoint device system 250 also includes an oscillator (e.g., a crystal oscillator) 272 that is configured to generate a stable reference clock signal 273 for the endpoint system memory 274 and the clock generator 268. In Figure 2In the example, the clock generator 224 at the host system 210 is configured to generate a stable reference clock signal, which is forwarded by the PHYRX block 226 to the endpoint device system 250 via the differential clock line 288. At the endpoint device system 250, the PHY RX block 270 receives the endpoint (EP) reference clock signal on the differential clock line 288 and forwards the EP reference clock signal to the clock generator 268. The EP reference clock signal may have a frequency of 100 MHz, but is not limited to this frequency. The clock generator 268 may be configured to generate a plurality of clock signals based on the EP reference clock signal from the differential clock line 288, as further discussed below. In this regard, the clock generator 268 may include a plurality of phase-locked loops (PLLs), where each PLL generates a corresponding one of the plurality of clock signals by multiplying the frequency of the EP reference clock signal.

[0037] The system 205 also includes a power management integrated circuit (PMIC) 290 coupled to a power supply 292 (e.g., mains voltage, battery, or other power source). The PMIC 290 is configured to convert the voltage of the power supply 292 into a plurality of supply voltages (e.g., using a switching regulator, a linear regulator, or any combination thereof). In this example, the PMIC 290 generates a voltage 242 for the oscillator 230, a voltage 244 for the PCIe controller 218, and a voltage 246 for the PHY TX block 222, the PHY RX block 226, and the clock generator 224. The voltages 242, 244, and 246 may be programmable, where the PMIC 290 is configured to set the voltage levels (corners) of the voltages 242, 244, and 246 according to instructions (e.g., from the host controller 212).

[0038] The PMIC 290 also generates a voltage 280 for the oscillator 272, a voltage 278 for the PCIe controller 262, and a voltage 276 for the PHY TX block 266, the PHY RX block 270, and the clock generator 268. The voltages 280, 278, and 276 may be programmable, where the PMIC 290 is configured to set the voltage levels (corners) of the voltages 280, 278, and 276 according to instructions (e.g., from the device controller 252). The PMIC 290 may be implemented on one or more chips. Although the PMIC 290 is shown as one PMIC in Figure 2 it should be understood that the PMIC 290 may be implemented by two or more PMICs. For example, the PMIC 290 may include a first PMIC for generating the voltages 242, 244, and 246 and a second PMIC for generating the voltages 280, 278, and 276. In this example, both the first PMIC and the second PMIC may be coupled to the same power supply 292 or different power supplies.

[0039] During operation, the PCIe interface circuit 216 on the host system 210 can send data from one or more host clients 214 to the endpoint device system 250 via the PCIe link 285. When the host controller negotiates the bandwidth for the link, data from one or more host clients 214 can be directed to the PCIe interface circuit 216 according to the PCIe mapping established by the host controller 212 during the initial configuration (sometimes referred to as link initialization). At the PCIe interface circuit 216, the PCIe controller 218 can perform transaction layer and data link layer functions on the data, such as packetizing the data, generating error correction codes to be sent with the data, and so on.

[0040] The PCIe controller 218 outputs the processed data to the PHY TX block 222 via the PIPE interface 220. The processed data includes data from one or more host clients 214 and overhead data (e.g., packet headers, error correction codes, etc.). In one example, the clock generator 224 can generate a clock 234 for an appropriate data rate or transmission rate based on the reference clock signal 232, and input the clock 234 to the PCIe controller 218 to time the operation of the PCIe controller 218. In this example, the PIPE interface 220 can include a 22-bit parallel bus that transmits 22 bits of data in parallel to the PHY TX block for each cycle of the clock 234. At a frequency of 250 MHz, the transmission rate is approximately 8 GT / s.

[0041] The PHY TX block 222 serializes the parallel data from the PCIe controller 218 and drives the PCIe link 285 with the serialized data. In this regard, the PHY TX block 222 can include one or more serializers and one or more drivers. The clock generator 224 can generate a high-frequency clock for the one or more serializers based on the reference clock signal 232.

[0042] At the endpoint device system 250, the PHY RX block 270 receives the serialized data via the PCIe link 285 and deserializes the received data into parallel data. In this regard, the PHY RX block 270 can include one or more receivers and one or more deserializers. The clock generator 268 can generate a high-frequency clock for the one or more deserializers based on the EP reference clock signal. The PHY RX block 270 transmits the deserialized data to the PCIe controller 262 via the PIPE interface 264. The PCIe controller 262 can recover the data from one or more host clients 214 from the deserialized data and forward the recovered data to one or more device clients 254.

[0043] On the endpoint device system 250, the PCIe interface circuit 260 may send data from one or more device clients 254 to the host system memory 240 via the PCIe link 285. In this regard, the PCIe controller 262 at the PCIe interface circuit 260 may perform transaction layer and data link layer functions on the data, such as packetizing the data, generating error correction codes to be sent along with the data, etc. The PCIe controller 262 outputs the processed data to the PHY TX block 266 via the PIPE interface 264. The processed data includes data from one or more device clients 254 and overhead data (e.g., packet headers, sequence numbers, error correction codes, etc.). An example of an error correction code is a cyclic redundancy check (CRC). In one example, the clock generator 268 may generate a clock based on the EP reference clock via the differential clock line 288 and input the clock to the PCIe controller 262 to control the timing operation of the PCIe controller 262.

[0044] The PHY TX block 266 serializes the parallel data from the PCIe controller 262 and drives the PCIe link 285 with the serialized data. In this regard, the PHY TX block 266 may include one or more serializers and one or more drivers. The clock generator 268 may generate a high-frequency clock for the one or more serializers based on the EP reference clock signal.

[0045] At the host system 210, the PHY RX block 226 receives the serialized data via the PCIe link 285 and deserializes the received data into parallel data. In this regard, the PHY RX block 226 may include one or more receivers and one or more deserializers. The clock generator 224 may generate a high-frequency clock for the one or more deserializers based on the reference clock signal 232. The PHY RX block 226 transfers the deserialized data to the PCIe controller 218 via the PIPE interface 220. The PCIe controller 218 may recover the data from one or more device clients 254 from the deserialized data and forward the recovered data to one or more host clients 214.

[0046] The communication mechanism between PCIe devices (e.g., host 210 and endpoint 250) involves three layers: the transaction layer, the data link layer, and the physical layer. Data packets processed at the transaction layer may be referred to herein as transaction layer packets (TLPs). A flow control mechanism may be used to control the transmission and retransmission of TLPs between PCIe devices, which may be referred to as link partners in the present disclosure. At the transmitter (e.g., the transmitting link partner), a sequence number is added to the header of each TLP transmitted. Also, an error checker (e.g., CRC) is added to each TLP. At the receiver (e.g., the receiving link partner), the TLPs are accepted and processed in the order of their sequence numbers. The TLPs with earlier sequence numbers are processed at the transaction layer before the TLPs with later sequence numbers can be presented to the transaction layer for processing. In the current PCIe implementation, if the sequence number of the received TLP is not the desired or expected sequence number, the TLP is discarded, and the receiver waits for another TLP with the desired sequence number. If the TLP has the correct sequence number, the receiver may present the TLP to the transaction layer for processing. Thus, frequent retransmission of TLPs may result in reduced throughput and increased power consumption. In addition, the current PCIe flow control implementation unnecessarily retransmits valid TLPs that pass the CRC check but are out of order. Thus, the PCIe link and link partners cannot enter a deep power saving state.

[0047] Figure 3 is a block diagram conceptually illustrating flow control of data packets (e.g., TLPs) between PCIe link partners. The link partners may include a transmitter 302 and a receiver 304. In one example, the transmitter 302 and the receiver 304 may be implemented as Figure 2 the host system 210 and the endpoint device 250. The transmitter 302 may send one or more TLPs to the receiver 304 via the PCIe link 306. At the transmitter, the data link layer 308 may provide the TLP to the replay buffer 310 and then send the TLP from the replay buffer via the PCIe link 306 to the receiver 304. Each TLP has a sequence number, e.g., between 0 and 4095. In some aspects, the sequence number 0 may be reserved for other flow control purposes, which will be described in more detail below. For example, for each TLP transmitted, the sequence number is incremented. When the sequence number reaches 4095, the sequence number rolls over to 1 and continues to increment again.

[0048] The replay buffer 310 may store a copy of each transmitted TLP for which feedback has not yet been received from the receiver 304. In one example, the replay buffer 310 may store copies of TLPs with sequence numbers 4094, 4095, 0, 1, and 2 that have been sent by the link 306 to the receiver 304. In this case, sequence number 4094 is the earliest, and sequence number 2 is the last in the sequence. When the receiver 304 receives a TLP, it checks for errors in the packet (e.g., CRC check), for example using the packet check block 314. The receiver also checks the sequence number of the TLP, for example using the packet check block 314 that may hold the Next Receive Sequence (NRS) number. The NRS number indicates the expected sequence number of the next TLP to be received. For example, the last correctly received TLP has a sequence number equal to NRS - 1. For example, after the receiver has correctly received a TLP with sequence number 4094, the NRS number is 4095. After each successful reception of a TLP, the receiver may increment the NRS.

[0049] When the receiver 304 receives a valid TLP with the correct sequence number (e.g., passing the CRC check), the receiver may process the TLP at its transaction layer 316, for example. In addition, the receiver may schedule an acknowledgment (ACK) for the TLP to be transmitted to the transmitter 302. For example, the receiver may use the ACK / NACK generator 318 to schedule the ACK. The ACK may indicate the sequence number of the TLP being acknowledged. When the transmitter 302 receives the ACK, the transmitter may check the sequence number associated with the ACK, for example using the ACK / NACK block 312. The transmitter may then clear the replay buffer 310 to remove copies of TLPs with sequence numbers less than or equal to the sequence number of the ACKed TLP.

[0050] When the receiver 304 receives a valid TLP (i.e., passes the CRC check) but has an invalid sequence number (i.e., greater than the NRS number or the expected sequence number), the receiver may store the valid TLP in the receive packet buffer (RPB) 320 for later processing instead of discarding the valid TLP. The receiver 304 may store multiple out-of-order valid TLPs in the RPB 320. In some aspects, the receiver 304 may transmit a NACK with parameters such that the transmitter 302 can selectively retransmit (replay) only those TLPs that were not successfully received (e.g., damaged or lost during transmission), rather than retransmitting (replaying) all the packets stored in the replay buffer 310. For example, the NACK may include a start sequence number (SSN) (first parameter) and an end sequence number (ESN) (second parameter) to indicate the range of packets that need to be retransmitted or replayed. For example, after receiving the NACK, the transmitter 302 retransmits all the TLPs in the replay buffer whose sequence numbers are between SSN + 1 and ESN (including SSN + 1 and ESN). In addition, the transmitter 302 clears all the TLPs in the replay buffer 310 whose sequence numbers are equal to or less than SSN.

[0051] In some aspects, the RPB 320 may be implemented as an array sorted in ascending or descending order of sequence numbers. Insertion sort techniques may be used to add new TLPs to the array to maintain the correct order of the TLPs in the array based on the sequence numbers of the TLPs. In some aspects, the RPB 320 may be implemented as a hash array. New TLPs may be added to the index position in the array corresponding to the sequence number of the TLP. In some aspects, a preselected sequence number (e.g., sequence number 0) may be reserved to notify the transmitter that the RPB is empty. In one example, numbers from 1 to 4095 may be used as TLP sequence numbers. For example, when the ESN in the NACK is set to 0, the NACK indicates that the RPB 320 is empty.

[0052] Figure 4 is a diagram illustrating a NACK packet format 400 that may be used to implement the above NACK. The NACK 400 includes an SSN field 402 and an ESN field 404 to indicate the SSN and ESN that can be used by the transmitter 302 to select the TLPs stored in the replay buffer 310 for retransmission. The NACK packet may also include a header 406 (e.g., an 8-bit header) indicating the packet type. The SSN field and the ESN field may each be 12 bits long. In some aspects, the NACK packet may be a data link layer packet (DLLP). In some aspects, according to the PCIe specification, the SSN 402 may correspond to reserved bits in the DLLP (e.g., 12 reserved bits).

[0053] Figure 5FIG. 500 is a flow chart illustrating a process 500 for selectively retransmitting TLPs at a transmitter. In one example, process 500 may be implemented at any PCIe device (e.g., transmitter 302) described herein. After receiving feedback from a link partner, at 502, the transmitter may verify whether the feedback is an ACK or NACK for a TLP previously transmitted to the link partner (e.g., receiver 304). At 504, when the feedback is an ACK, the transmitter may clear one or more TLPs in its replay buffer having sequence numbers equal to or earlier than the sequence number indicated by the ACK. For example, the ACK may include an SSN.

[0054] At 506, when the feedback is a NACK, the transmitter determines whether the receiver's receive packet buffer (RPB) (e.g., RPB 320) is empty based on the SSN and ESN included in the NACK. For example, a preselected value of the ESN (e.g., ESN = 0) may indicate that the receiver's RPB is empty. At 508, when the NACK indicates that the receiver's RPB is empty, the transmitter may retransmit (replay) all TPLs stored in its replay buffer (e.g., replay buffer 310). At 510, when the NACK indicates that the RPB is not empty (i.e., contains one or more TLPs), the transmitter may retransmit TLPs and / or clear TLPs from the replay buffer based on the SSN and ESN provided by the NACK. For example, the transmitter clears TLPs having sequence numbers earlier than or equal to the SSN and retransmits any TLPs having sequence numbers later than the SSN and equal to or earlier than the ESN.

[0055] Figure 6 FIG. 600 is a flow chart illustrating a process 600 for receiving TLPs at a receiver. In one example, process 600 may be implemented at any PCIe device (e.g., receiver 304) described herein. After receiving a TLP from a link partner, at 602, the receiver determines whether the packet is valid (i.e., passes a CRC check). At 604, the receiver determines whether the TLP has the correct or expected sequence number, e.g., the NRS number maintained at the receiver's packet check block 314. For example, if the TLP has a sequence number later than the NRS, it indicates that the TLP is out of order.

[0056] At 606, when the received TLP is valid and has the correct sequence number, the receiver may process the TLP at the transaction layer. At 608, the receiver may schedule an ACK to be sent along with the sequence number of the TLP. Then, the receiver may increment the NRS number for the next expected TLP. For example, the receiver may increment the NRS number maintained at the packet check block 314.

[0057] At 609, the receiver determines whether the sequence number of the received TLP is later than the NRS number. At 610, when the received TLP is valid and has a sequence number later than the expected sequence number, the receiver may add the received TLP to a buffer instead of discarding the out-of-order TLP. In one example, the buffer may be the RPB 320, which may store the TLP in a sorted array of its sequence numbers or at an index location corresponding to the sequence number of the TLP. Keeping the out-of-order but valid TLP in the buffer can reduce unnecessary retransmission of these TLPs.

[0058] At 612, the receiver checks the buffer status. At 614, when the buffer (e.g., RPB 320) is empty, the receiver may send a NACK with a special ESN (e.g., value 0) indicating that the buffer is empty. In response, the transmitter may retransmit all TLPs stored in the replay buffer 310. At 616, when the buffer (e.g., RPB 320) is not empty, if the buffer contains a TLP with the correct expected sequence number (e.g., the TLP with the latest sequence number or index value in the buffer), the receiver may dequeue the buffer to process one or more TLPs in order at the transaction layer. For each TLP dequeued from the buffer, the receiver increments the NRS number. The receiver repeats the dequeue process until there are no more TLPs to process at the transaction layer.

[0059] When no TLP in the buffer has the correct sequence number (i.e., NRS number), the receiver may send a NACK that includes information to facilitate packet retransmission at the transmitter. For example, the NACK may include the SSN and ESN as described above to request the transmitter to retransmit the lost or damaged packet.

[0060] Figure 7FIG. is an illustration showing an example of selective TLP retransmission according to some aspects. In one example, a transmitter 702 is configured to send several TLPs to a receiver 704. The transmitter 702 and the receiver 704 can be any PCIe device described herein, for example, the transmitter 302 and the receiver 304. At 706, the transmitter sends four packets (e.g., TLPs) to the receiver. The transmitter may store copies of the sent packets #1 to #4 in a replay buffer 708 (e.g., the replay buffer 310). In this case, the receiver successfully receives packets #1, #3, and #4, but fails to receive packet #2. The receiver processes packet #1 at the transaction layer, but stores packets #3 and #4 in a received packet buffer 710 (e.g., RPB 320) because packets #3 and #4 are not the expected packet #2. Then, at 712, the receiver transmits a NACK(1,2) to request the transmitter to retransmit packet #2 and indicates that the transmitter may clear packet #1 and earlier packets (if any) stored in the replay buffer 708.

[0061] At 714, the transmitter retransmits packet #2, and the receiver successfully receives packet #2. After receiving packet #2, the receiver may process packets #2, #3, and #4 stored in the received packet buffer 710. At this time, the received packet buffer 710 dequeues and becomes empty. Meanwhile, the transmitter may send packets #5, #6, #7, and #8 to the receiver. In this case, the receiver fails to receive packet #5, but correctly receives packets #6, #7, and #8. Therefore, the receiver may store packets #6, #7, and #8 in the received packet buffer 710 for later processing.

[0062] At 716, the transmitter transmits a NACK(4,5) to request the transmitter to retransmit packet #5 and indicates that the transmitter may clear packet #4 and earlier packets (if any) stored in the replay buffer 708. In response, the transmitter deletes the copies of packets #2, #3, and #4 in the replay buffer 708. At 718, the transmitter retransmits packet #5. This time, the receiver correctly receives packet #5 and processes packets #5, #6, #7, and #8 at the transaction layer. At this time, the received packet buffer 710 dequeues and becomes empty. Then, the receiver sends an ACK(8) to the transmitter. After receiving the ACK(8), the transmitter may clear any packets in the replay buffer 708 whose sequence numbers are equal to or earlier than #8 (e.g., packets #5, #6, #7, and #8).

[0063] In Figure 7 the example shown, the transmitter sends eight packets (packets #1 to #8) to the receiver. In the case of losing two packets (packets #2 and #5), the transmitter needs to use the above regarding Figures 3 to 6The described selective retransmission technique is used to send 10 packets, including 2 retransmissions. In contrast, the current PCIe implementation will cause the transmitter to send 15 packets, including 7 retransmissions. For example, according to the current PCIe implementation, the transmitter may retransmit packets #2, #3, and #4 after receiving a NACK for packet #2, and retransmit packets #5 to #8 after receiving a NACK for packet #5. In this case, more packets are retransmitted compared to using the techniques of the present disclosure.

[0064] Figure 8 is a block diagram of a link interface processing circuit. The processing circuit 804 is a device that can be part of a host or an endpoint. The processing circuit is coupled to a link 802 having a plurality of duplex channels similar to the duplex channels described with respect to Figures 5 to 7 a PCIe link, for example. The link 802 can be coupled to another PCIe device (e.g., an endpoint or a host) at the opposite end. Data and control information conveyed as packets through the link 802 are coupled to a link interface 820 (e.g., a PCIe interface), which provides a PHY-level interface to the link 802 and converts baseband signals into packets. The data and control packets are transferred via the link interface 820 through a bus 810 to other components of the processing circuit 804. The link interface 820 has a direct connection to an interface configuration circuit 818 for configuration and control settings for the operation of the link 802.

[0065] The processing circuit 804 further includes a memory 821, which can be used to store data and information used by the processor during various operations. In some aspects, the memory 821 can store information and data packets for flow control of TLPs. In one example, the memory 821 can provide a buffer (e.g., a replay buffer 310) for storing copies of the transmitted TLPs. In one example, the memory 821 can provide a buffer (e.g., a received packet buffer 320) for storing the received TLPs.

[0066] The processing circuit 804 further includes a timer circuit 812 coupled to the bus 810. The timer circuit 812 can be configured for various timing-related functions, such as timing for latency, inactivity, acknowledgment, and flow control. The timer circuit 812 can access a computer-readable storage medium 808 to access code 832 for managing the timer. In some aspects, the storage medium is a non-transitory computer-readable medium. The timer circuit 812 can also access registers maintained in the storage medium 808 (and / or the memory 821), which contain receive (RX) traffic timing thresholds 834 and transmit (TX) traffic timing thresholds 836 that can be used during flow control of TLPs.

[0067] The processing circuit 804 may also include a power management circuit 814 that manages the power to each line / channel of the link 802 and to other components of the processing circuit 804. The power management circuit 814 accesses, via a bus 810, code 840 for managing PCIe power, as well as a transmit line status register 842 and a receive line status register 844. These registers may be used to store the status of each transmit line and each receive line, or the transmit side and the receive side of the link. This status may be determined using code 832 for managing timers, code 840 for managing PCIe power, or in another way.

[0068] The processing circuit 804 may also include a link flow control circuit 816 that monitors traffic (e.g., TLPs) on the link 802 and provides flow control for that traffic. For example, the link flow control circuit 816 may monitor transmitted TLPs and / or received TLPs to provide flow control for the link. In one example, the link flow control circuit may generate ACK / NACK for TLPs. The link flow control circuit 816 accesses code 850 for link flow control in a storage medium 808 and also accesses registers to store results and obtain traffic activity information for link flow control. For example, transmit traffic activity 852 and receive traffic activity 854 may be used to monitor transmit traffic activity and receive traffic activity, respectively.

[0069] The power management circuit 814 may manage the power of the transmit lines and the power of the receive lines based on the transmit traffic activity and the receive traffic activity. The interface configuration circuit 818 may modify the configuration in response to the power management circuit 814. For example, the interface configuration circuit 818 may change the link state of the link 802.

[0070] The interface configuration circuit 818 is coupled to the bus 810, like the link flow control circuit 816, the power management circuit 814, and the timer circuit 812, such that each of these blocks may communicate with each other, with the storage medium 808, and with the processor 806. The processor 806 may control the operation of other components and appropriately initiate instances of each component or its functions based on the operation of the processing circuit 804. The interface configuration circuit 818 also accesses code 860 for configuring the PCIe interface. When executing this code, the interface configuration circuit 818 may read and write values from various configuration registers. For example, these registers include a TX control, status, and capability register 862 and an RX control, status, and capability register 864. These registers may be accessed and read at the start of link initialization and then updated with the results of the initialization. The registers may also be modified in response to power management and bandwidth negotiation or to change the state of one or more transmit lines or receive lines of the link 802.

[0071] The processing circuit 804 can initialize the link 802, manage power, link state, and change the number of active lanes of the link 802. In operation, bandwidth requests may also be received from a host or an endpoint. The bandwidth requests may cause bandwidth negotiation, and subsequently change the values set to the control, status, and capability registers. Then, the number of active lanes may be changed in response to transmit traffic activity and receive traffic activity. The link flow control circuit 816 can also monitor the TX traffic activity of the transmit lanes of the link 802, and monitor the RX traffic activity of the receive lanes of the link 802. The TX traffic activity and the RX traffic activity are evaluated to determine a change in the number of active lanes. The power management circuit 814 can change the link state of one or more TX lanes or RX lanes. Then the state change can be recorded in the TX lane status register 842 and the RX lane status register 844. This evaluation can be performed in different ways. In some examples, at the link flow control circuit 816, the TX traffic activity is compared with one or more thresholds in the TX traffic register 852, and the RX traffic activity is compared with one or more thresholds in the RX traffic register 854. Then a message can be transmitted through the link 802 to the connected device (e.g., a host or an endpoint).

[0072] After changing the number of active lanes or the link state, the power management 814 can change the voltage level of one or more of the voltages 276, 278, and 280 supplied by the PMIC 290 by commanding the PMIC 290 to set the voltage level of one or more of the voltages supplied by the PMIC 290, as Figure 2 shown. The power management circuit 814 can also connect or disconnect the power to the drivers and receivers of the affected lanes according to the new number of active lanes. For example, if the number of active lanes is reduced, the power management circuit 814 can power off the drivers in the PHY TX block 222 and / or the receivers in the PHY RX block 226 corresponding to the lanes in the link 802 that are deactivated due to the change. The power management circuit 814 can power off the selected drivers and / or receivers by transmitting instructions to the power switch circuit to turn off the selected drivers and / or receivers. Thus, power is managed according to the negotiated bandwidth by providing one or more voltages to the interface circuit of the link and by setting the level of the one or more voltages.

[0073] Figure 9 FIG. 900 is a flow chart of a method 900 for flow control of a wired data link (e.g., a PCIe link) in accordance with aspects of the present disclosure. In certain aspects, the method 900 provides techniques for flow control of a wired data link. As described herein, the wired data link can be a PCIe link, however the method can be adapted to other data links that use flow control.

[0074] At 902, the method includes the process of sending one or more first data packets (e.g., to a link partner such as a host or an endpoint) via a wired data link (e.g., a PCIe link). In one aspect, the PCIe interface 820 may provide the component for sending one or more first data packets using the wired data link. For example, one or more first data packets may be Figure 7 data packets #1, #2, #3, and #4. Each of these data packets has a sequence number to facilitate flow control and reordering of the data packets at the receiver. In one aspect, the link flow control circuit 816 may be the component that provides flow control-related functions for the wired data link (e.g., a PCIe link). In some aspects, the device may store a copy of the sent first data packets in a buffer (e.g., the replay buffer 310), which may be implemented using Figure 8 the memory 821 and / or the storage medium 808.

[0075] At 904, the method includes the process of receiving negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets in the first data packets. For example, one or more of the first data packets in the first data packets may be corrupted or lost, such that the device may request retransmission of these data packets. In one aspect, the PCIe interface 820 may provide the component for receiving negative feedback from the link partner. In some aspects, the negative feedback may be a NACK (e.g., NACK 712, 716), as described above with respect to Figures 3 to 7 . In one example, the negative feedback may be Figure 4 a NACK 400 including a first parameter (e.g., SSN 402) and a second parameter (e.g., ESN 404). In one aspect, the link flow control circuit 816 may provide the component for processing the negative feedback (e.g., a NACK) to determine the packets requested for retransmission. For example, the negative feedback may be NACK(1,2), which requests retransmission of data packet #2 instead of data packets #3 and #4 because these packets have been stored in the received packet buffer 320.

[0076] At 906, the method includes a process of retransmitting one or more requested first data packets. In one aspect, the PCIe interface 820 may provide a component for retransmitting the requested first data packets. In one aspect, the link flow control circuit 816 may provide the requested data packets for retransmission to the PCIe interface 820. For example, the link flow control circuit 816 may retrieve a copy of the retransmitted data packets from a replay buffer maintained in the memory 821 and / or the storage medium 808. For example, the method may retransmit data packet #2, but skip data packets #3 and #4. Then, the method may transmit data packets #5, #6, and so on.

[0077] Figure 10 FIG. 1000 is a flowchart of a method 1000 for flow control of a wired data link (e.g., a PCIe link) in accordance with aspects of the present disclosure. In certain aspects, method 1000 provides techniques for flow control of a wired data link. As described herein, the data link may be a PCIe link, however the method may be adapted to other data links that use flow control.

[0078] At 1002, the method includes a process of receiving one or more first data packets from a link partner (e.g., a host or an endpoint) via a wired data link (e.g., a PCIe link). The one or more first data packets are included in an expected sequence of data packets. In one aspect, the PCIe interface 820 may provide a component for receiving the one or more first data packets using the PCIe link. For example, the one or more first data packets may be Figure 7 data packets #1, #2, #3, and #4. Each of these data packets has a sequence number to facilitate flow control and reordering of the data packets. In one aspect, the link flow control circuit 816 may provide a component for providing flow control related functions for the data packets. In some aspects, the device may store a copy of the received first data packets in a receive packet buffer 320 (e.g., implemented using Figure 8 the memory 821 and / or the storage medium 808). The received data packets may be valid (e.g., pass a CRC check), but do not have the correct sequence number for future processing at the transaction layer. For example, the received valid data packets may have a sequence number that is later in time (e.g., greater than) the expected sequence number.

[0079] At 1004, the method includes a process of sending negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more second data packets, the one or more second data packets being included in an expected sequence of data packets. For example, at least one of the second data packets has a sequence number that is earlier in time (e.g., greater) than the sequence numbers of one or more first data packets. At least one of the second data packets may have a sequence number equal to the expected sequence number. In one aspect, the PCIe interface 820 may provide components for sending negative feedback using the PCIe link 802. In some aspects, the negative feedback may be a NACK, as described above with respect to Figures 3 to 7 as described. In one example, the negative feedback may be Figure 4 a NACK 400 including a first parameter (e.g., SSN 402) and a second parameter (e.g., ESN 404). In one aspect, the link flow control circuit 816 may provide components for generating negative feedback (e.g., NACK) based on the sequence number of the received first packet and the expected sequence number, the expected sequence number indicating the next packet to be received in the correct sequence number order for transaction layer processing.

[0080] At 1006, the method includes a process of receiving one or more second data packets. In one aspect, the PCIe interface 820 may provide components for receiving second data packets from the PCIe link 802. In one aspect, the link flow control circuit 816 may process the received second data packets based on the sequence numbers of the data packets. In one example, the link flow control circuit 816 may store the second data packets at the receive packet buffer 320 (e.g., implemented using Figure 8 the memory 821 and / or the storage medium 808). In one example, if at least one of the second data packets has a sequence number equal to the expected sequence number, the link flow control circuit 816 may process the received data packets and / or the second data packets at the transaction processing layer based on their respective sequence numbers.

[0081] An overview of various embodiments of the present disclosure is provided below.

[0082] Embodiment 1: A method of operating a device for data communication, the method comprising: sending one or more first data packets to a link partner via a wired data link; receiving negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets; and retransmitting the requested one or more first data packets.

[0083] Embodiment 2: The method according to Embodiment 1, the method further comprising: sending a second data packet after the retransmission of the requested one or more first data packets, wherein the sequence number of the second data packet is disordered with respect to the corresponding sequence numbers of the retransmitted one or more first data packets.

[0084] Embodiment 3: The method according to Embodiment 1, the method further comprising: storing the one or more first data packets in a replay buffer, and after receiving the negative feedback, clearing one or more of the stored first data packets from the replay buffer according to the indication of the first parameter.

[0085] Embodiment 4: The method according to Embodiment 3, wherein retransmitting the requested one or more first data packets comprises: retransmitting one or more of the first data packets stored in the replay buffer determined based on the first parameter and the second parameter.

[0086] Embodiment 5: The method according to Embodiment 1, 2, 3 or 4, wherein the first parameter of the negative feedback is configured to indicate successful transmission of one or more of the first data packets, each first data packet having a sequence number that is earlier than or equal to the sequence number indicated by the first parameter in time.

[0087] Embodiment 6: The method according to Embodiment 1, 2, 3 or 4, wherein the first parameter includes a first sequence number, and the second parameter includes a second sequence number that is later than the first sequence number in time, and wherein each of the retransmitted first data packets includes a sequence number that is later than the first sequence number and earlier than or equal to the second sequence number in time.

[0088] Embodiment 7: The method according to Embodiment 1, 2, 3 or 4, wherein the second parameter includes a value reserved to indicate that the receive buffer of the link partner is empty.

[0089] Embodiment 8: The method according to Embodiment 1, 2, 3 or 4, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

[0090] Embodiment 9: A method of operating a device for data communication, the method comprising: receiving, via a wired data link, one or more first data packets from a link partner, the one or more first data packets being included in an expected sequence of data packets; transmitting negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request a retransmission of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier in time than the corresponding sequence numbers of the one or more first data packets; and receiving the retransmission of the one or more second data packets.

[0091] Embodiment 10: The method according to Embodiment 9, the method further comprising: receiving a third data packet after receiving the one or more second data packets, wherein the sequence number of the third data packet is unordered relative to the corresponding sequence numbers of the one or more second data packets.

[0092] Embodiment 11: The method according to Embodiment 9, the method further comprising: storing the one or more first data packets in a receive buffer before processing the one or more first data packets at a transaction layer, wherein the corresponding sequence numbers of the one or more first data packets are later in time than an expected sequence number.

[0093] Embodiment 12: The method according to Embodiment 11, wherein the receive buffer is configured to store the one or more first data packets in an order according to the corresponding sequence numbers of the one or more first data packets.

[0094] Embodiment 13: The method according to Embodiment 11 or 12, the method further comprising: dequeuing, in response to receiving the one or more second data packets, one or more of the first data packets from the receive buffer; processing the dequeued one or more first data packets at the transaction layer; and processing the one or more second data packets at the transaction layer after processing the dequeued one or more first data packets.

[0095] Embodiment 14: The method according to Embodiment 13, the method further comprising: incrementing the expected sequence number after dequeuing each first data packet, wherein each dequeued first data packet has a sequence number equal to the expected sequence number.

[0096] Embodiment 15: The method according to Embodiment 11 or 12, wherein the second parameter includes a value reserved to indicate that the receive buffer is empty.

[0097] Example 16: The method according to Example 9, 10, 11, or 12, wherein: the first parameter includes a first sequence number, and the second parameter includes a second sequence number that is earlier in time than the first sequence number; and the one or more second data packets have corresponding sequence numbers that are later in time than the first sequence number and earlier in time than the second sequence number or equal to the second sequence number.

[0098] Example 17: The method according to Example 9, 10, 11, or 12, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

[0099] Example 18: A device for data communication, the device including: an interface circuit configured to provide an interface to a wired data link connected to a link partner; and a controller configured to: transmit one or more first data packets to the link partner via the wired data link; receive negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets; and retransmit the requested one or more first data packets.

[0100] Example 19: The device according to Example 18, wherein the controller is further configured to: transmit a second data packet after the retransmission of the requested one or more first data packets, wherein the sequence number of the second data packet is unordered relative to the corresponding sequence numbers of the one or more first data packets being retransmitted.

[0101] Example 20: The device according to Example 18, the device further including: a replay buffer, wherein the controller is further configured to: store the one or more first data packets in the replay buffer; and, after receiving the negative feedback, clear one or more of the stored first data packets from the replay buffer according to an indication of the first parameter.

[0102] Example 21: The device according to Example 20, wherein, for retransmitting the requested one or more first data packets, the controller is further configured to: retransmit one or more of the first data packets stored in the replay buffer determined based on the first parameter and the second parameter.

[0103] Example 22: The apparatus according to any one of Examples 18, 19, 20, or 21, wherein the first parameter of the negative feedback is configured to indicate successful transmission of one or more first data packets in the first data packet, each first data packet having a sequence number that is earlier than or equal to the sequence number indicated by the first parameter in time.

[0104] Example 23: The apparatus according to any one of Examples 18, 19, 20, or 21, wherein the first parameter includes a first sequence number, and the second parameter includes a second sequence number that is later than the first sequence number in time, and wherein each first data packet of the retransmitted first data packets includes a sequence number that is later than the first sequence number and earlier than or equal to the second sequence number in time.

[0105] Example 24: The apparatus according to any one of Examples 18, 19, 20, or 21, wherein the second parameter includes a value reserved to indicate that the receive buffer of the link partner is empty.

[0106] Example 25: The apparatus according to any one of Examples 18, 19, 20, or 21, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

[0107] Example 26: An apparatus for data communication, the apparatus comprising: an interface circuit configured to provide an interface to a wired data link connected to a link partner; and a controller configured to: receive, via the wired data link, one or more first data packets from the link partner, the one or more first data packets being included in an expected sequence of data packets; transmit negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier than the corresponding sequence number of the one or more first data packets in time; and receive the retransmission of the one or more second data packets.

[0108] Example 27: The apparatus according to Example 26, wherein the controller is further configured to: receive a third data packet after receiving the one or more second data packets, wherein the sequence number of the third data packet is disordered with respect to the corresponding sequence numbers of the one or more second data packets.

[0109] Example 28: The apparatus according to Example 26, wherein the controller is further configured to: store the one or more first data packets in a receive buffer before processing the one or more first data packets at the transaction layer, wherein the respective sequence numbers of the one or more first data packets are later in time than an expected sequence number.

[0110] Example 29: The apparatus according to Example 28, wherein the receive buffer is configured to store the one or more first data packets in an order according to the respective sequence numbers of the one or more first data packets.

[0111] Example 30: The apparatus according to Example 28 or 29, wherein the controller is further configured to: dequeue one or more of the first data packets from the receive buffer in response to receiving the one or more second data packets; process the dequeued one or more first data packets at the transaction layer; and process the one or more second data packets at the transaction layer after processing the dequeued one or more first data packets.

[0112] Example 31: The apparatus according to Example 30, wherein the controller is further configured to: increment the expected sequence number after dequeuing each first data packet, wherein each dequeued first data packet has a sequence number equal to the expected sequence number.

[0113] Example 32: The apparatus according to Example 28 or 29, wherein the second parameter includes a value reserved to indicate that the receive buffer is empty.

[0114] Example 33: The apparatus according to Example 26, 27, 28 or 29, wherein: the first parameter includes a first sequence number, and the second parameter includes a second sequence number earlier in time than the first sequence number; and the one or more second data packets have respective sequence numbers later in time than the first sequence number and earlier in time than the second sequence number or equal to the second sequence number.

[0115] Example 34: The apparatus according to Example 26, 27, 28 or 29, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

[0116] It should be understood that the present disclosure is not limited to the exemplary terms used above to describe aspects of the present disclosure. For example, bandwidth may also be referred to as throughput, data rate, or another term.

[0117] Although aspects of the present disclosure have been discussed above using an example of the PCIe standard, it should be understood that the present disclosure is not limited to this example and may be used with other standards.

[0118] Each of the host client 214, host controller 212, device controller 252, and device client 254 discussed above may be implemented using a controller or processor configured to perform the functions by executing software including code for performing the functions described herein. The software may be stored on a non-transitory computer-readable storage medium, such as RAM, ROM, EEPROM, optical disk, and / or magnetic disk, shown as host system memory 240, endpoint system memory 274, or another memory.

[0119] Any reference in this document to an element using, for example, "first", "second", etc. generally does not limit the quantity or order of those elements. Rather, these references are used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to a first element and a second element does not mean that only two elements can be employed, or that the first element must be located before the second element.

[0120] Within this disclosure, the term "exemplary" is used to mean "serving as an example, instance, or illustration". Any particular implementation or aspect described herein as "exemplary" need not be construed as superior to or better than other aspects of the disclosure. Similarly, the term "aspect" does not require that all aspects of the disclosure include the discussed feature, advantage, or mode of operation. The term "coupled" is used herein to refer to a direct or indirect electrical coupling or other communication coupling between two structures. Additionally, the term "about" means within ten percent of the stated value.

[0121] The foregoing description of the disclosure has been provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of operating a device for data communication, the method comprises: sending one or more first data packets to a link partner via a wired data link; receiving negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets in the first data packets; and retransmitting the requested one or more first data packets.

2. The method according to claim 1, the method further comprises: sending a second data packet after the retransmission of the requested one or more first data packets, wherein the sequence number of the second data packet is unordered relative to the corresponding sequence numbers of the retransmitted one or more first data packets.

3. The method according to claim 1, the method further comprises: storing the one or more first data packets in a replay buffer, and after receiving the negative feedback, clearing one or more of the stored first data packets from the replay buffer according to the indication of the first parameter.

4. The method according to claim 3, wherein retransmitting the requested one or more first data packets comprises: retransmitting one or more of the first data packets stored in the replay buffer determined based on the first parameter and the second parameter.

5. The method according to claim 1, wherein the first parameter of the negative feedback is configured to indicate successful transmission of one or more of the first data packets, each first data packet having a sequence number that is earlier than or equal to the sequence number indicated by the first parameter in time.

6. The method according to claim 1, wherein the first parameter includes a first sequence number, and the second parameter includes a second sequence number that is later than the first sequence number in time, and wherein each of the retransmitted first data packets includes a sequence number that is later than the first sequence number and earlier than or equal to the second sequence number in time.

7. The method according to claim 1, wherein the second parameter includes a value reserved to indicate that the receive buffer of the link partner is empty.

8. The method according to claim 1, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

9. A method of operating a device for data communication, the method comprises: receiving one or more first data packets from a link partner via a wired data link, the one or more first data packets being included in an expected sequence of data packets; sending negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier than the corresponding sequence numbers of the one or more first data packets in time; and receiving the retransmission of the one or more second data packets.

10. The method according to claim 9, the method further comprises: receiving a third data packet after receiving the one or more second data packets, wherein the sequence number of the third data packet is disordered relative to the corresponding sequence numbers of the one or more second data packets.

11. The method according to claim 9, the method further comprises: storing the one or more first data packets in a receive buffer before processing the one or more first data packets at a transaction layer, wherein the corresponding sequence numbers of the one or more first data packets are later in time than an expected sequence number.

12. The method according to claim 11, wherein the receive buffer is configured to store the one or more first data packets in an order according to the corresponding sequence numbers of the one or more first data packets.

13. The method according to claim 11, the method further comprises: dequeuing one or more of the first data packets from the receive buffer in response to receiving the one or more second data packets; processing the dequeued one or more first data packets at the transaction layer; and processing the one or more second data packets at the transaction layer after processing the dequeued one or more first data packets.

14. The method according to claim 13, the method further comprises: incrementing the expected sequence number after dequeuing each first data packet, wherein each dequeued first data packet has a sequence number equal to the expected sequence number.

15. The method according to claim 11, wherein the second parameter includes a value reserved to indicate that the receive buffer is empty.

16. The method according to claim 9, wherein: the first parameter includes a first sequence number, and the second parameter includes a second sequence number earlier in time than the first sequence number; and the one or more second data packets have corresponding sequence numbers later in time than the first sequence number and earlier in time than the second sequence number or equal to the second sequence number.

17. The method according to claim 9, wherein the wired data link includes a Peripheral Component Interconnect Express (PCIe) link.

18. A device for data communication, the device comprises: an interface circuit configured to provide an interface to a wired data link connected to a link partner; and a controller configured to: send one or more first data packets to the link partner via the wired data link; receive negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request retransmission of one or more of the first data packets; and retransmit the requested one or more first data packets.

19. The device according to claim 18, wherein the controller is further configured to: After the resending of the one or more first data packets requested, a second data packet is sent, wherein the sequence number of the second data packet is disordered relative to the corresponding sequence numbers of the one or more first data packets being resent.

20. The apparatus according to claim 18, the apparatus further comprises: a replay buffer, wherein the controller is further configured to: store the one or more first data packets in the replay buffer; after receiving the negative feedback, according to the indication of the first parameter, clear one or more of the stored first data packets from the replay buffer; and resend one or more of the first data packets stored in the replay buffer determined based on the first parameter and the second parameter.

21. The apparatus according to claim 18, wherein the first parameter of the negative feedback is configured to indicate successful transmission of one or more of the first data packets, each first data packet having a sequence number that is earlier than or equal to the sequence number indicated by the first parameter in time.

22. The apparatus according to claim 18, wherein the first parameter includes a first sequence number, and the second parameter includes a second sequence number that is later than the first sequence number in time, and wherein each of the first data packets being resent includes a sequence number that is later than the first sequence number and earlier than or equal to the second sequence number in time.

23. The apparatus according to claim 18, wherein the second parameter includes a value reserved to indicate that the receive buffer of the link partner is empty.

24. An apparatus for data communication, the apparatus comprises: an interface circuit configured to provide an interface to a wired data link connected to a link partner; and a controller configured to: receive, via the wired data link, one or more first data packets from the link partner, the one or more first data packets being included in an expected sequence of data packets; send a negative feedback including a first parameter and a second parameter, the first parameter and the second parameter being configured to request a resend of one or more second data packets included in the expected sequence of data packets, at least one of the second data packets having a sequence number that is earlier than the corresponding sequence numbers of the one or more first data packets in time; and receive the resend of the one or more second data packets.

25. The apparatus according to claim 24, wherein the controller is further configured to: receive a third data packet after receiving the one or more second data packets, wherein the sequence number of the third data packet is disordered relative to the corresponding sequence numbers of the one or more second data packets.

26. The apparatus according to claim 24, wherein the controller is further configured to: Store the one or more first data packets in a receive buffer before processing the one or more first data packets at a transaction layer, wherein the respective sequence numbers of the one or more first data packets are later in time than an expected sequence number.

27. The apparatus according to claim 26, wherein the receive buffer is configured to store the one or more first data packets in an order according to the respective sequence numbers of the one or more first data packets.

28. The apparatus according to claim 26, wherein the controller is further configured to: Dequeue one or more first data packets from the receive buffer in response to receiving the one or more second data packets; Process the dequeued one or more first data packets at the transaction layer; Process the one or more second data packets at the transaction layer after processing the dequeued one or more first data packets; And Increment the expected sequence number after dequeuing each first data packet, wherein each dequeued first data packet has a sequence number equal to the expected sequence number.

29. The apparatus according to claim 26, wherein the second parameter includes a value reserved to indicate that the receive buffer is empty.

30. The apparatus according to claim 24, Wherein: The first parameter includes a first sequence number, and the second parameter includes a second sequence number earlier in time than the first sequence number; and The one or more second data packets have respective sequence numbers later in time than the first sequence number and earlier in time than the second sequence number or equal to the second sequence number.

Images