Adaptive replay of packets in peripheral interface links

Abstract

The present invention provides a system, method, and circuit for adaptive replay of data link packets in peripheral interfaces. [Solution] In a computer system, the data link layer, which follows the peripheral interface protocol of the device, includes a replay buffer memory to support both sequential and selective replay in response to packet errors. Upon detecting an erroneous packet, the receiver determines an adaptive threshold for selecting a replay mode based on a set of link operation parameters, and then selects between selective and sequential replay mode by comparing the fill level of the replay buffer with the adaptive threshold determined for that packet error.

Description

Technical Field

[0001] This disclosure generally relates to data link layer communication via a peripheral interface, and more particularly to using an adaptive replay to dynamically select from replay modes to handle packet errors.

Background Art

[0002] PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard widely used in computer systems to connect various components and peripherals such as data storage devices. As data transfer speeds continue to increase, it is becoming increasingly difficult to maintain reliable communication between devices. PCIe incorporates error detection and correction mechanisms, including packet replay, to ensure data integrity. However, conventional replay methods can lead to inefficiencies in bandwidth utilization and increased latency, especially when only a small portion of the transmitted data is affected by errors. For example, selective replay of packets can often be more efficient than sequential replay of all packets, but selective replay can be limited by the size of the replay buffer on the receiving side of the data link layer of the PCIe interface. Under some operating conditions, it can be advantageous to proactively select between selective replay and sequential replay. As computing systems demand higher performance and lower latency, there is an increasing need for more advanced and adaptive techniques for managing packet replay in PCIe communication.

[0003] Therefore, there is still a need for more advanced and adaptive techniques for managing data link packet replay in PCIe communication that can improve bandwidth utilization while maintaining data integrity and reducing latency.

Summary of the Invention

[0004] Various embodiments for adaptive replay of data link packets in peripheral interfaces are described. More specifically, the replay logic circuit dynamically determines an adaptive threshold based on link operation parameters and selects between selective and sequential replay modes by comparing the replay buffer fill level with the adaptive threshold, enabling efficient handling of packet errors while optimizing buffer usage and system performance.

[0005] One common embodiment includes a device that includes a peripheral interface configured to connect to a host system using a peripheral interface protocol. The peripheral interface protocol includes a data link layer configured to send and receive data link packets, and the peripheral interface includes a replay buffer memory configured to store received data link packets following an erroneous packet, and a replay logic circuit, which is configured to determine an error in a data link packet received via the peripheral interface, dynamically determine an adaptive threshold for selecting a replay mode based on a set of link operation parameters, and, based on comparing the fill level of the replay buffer memory with the adaptive threshold, select a replay mode to be selected from a selective replay mode for requesting retransmission of only the erroneous data link packet and a sequential replay mode for requesting retransmission of the erroneous data link packet and subsequent data link packets sent after the erroneous data link packet, and, based on the selected replay mode, send a replay request to the host system.

[0006] The implementation may include one or more of the following features. The replay logic circuit may be further configured to store error-free sequential data link packets received after an errored data link packet in a replay buffer memory, depending on the selective replay mode, and to sequentially transmit the transaction layer content from the error-free version of the errored data link packet and the error-free sequential data link packet in the replay buffer memory, depending on the host system's reception of the error-free version of the errored data link packet, depending on the sequential replay mode, depending on the sequential replay mode, and to sequentially transmit the transaction layer content from the error-free version of the errored data link packet and the retransmitted sequential data link packet, depending on the reception of the error-free version of the errored data link packet and the error-free version of the retransmitted sequential data link packet. The replay buffer memory may have a buffer size corresponding to the total number of data link packets that can be stored in the replay buffer memory at one time, the fill level may correspond to the current number of data link packets stored in the replay buffer memory for pending selective replay requests, the adaptive threshold may correspond to the maximum fill level smaller than the buffer size of the replay buffer memory, and the replay logic circuit may be further configured to periodically determine the adaptive threshold from a set of maximum fill levels based on a set of link operation parameters. The peripheral interface protocol may support a flow control mode configured to change the memory size and format of data link packets, and the replay logic circuit may be further configured to determine that the flow control mode is active, determine the fill level of the replay buffer memory based on the memory size and format of data link packets in the flow control mode, and dynamically determine the adaptive threshold based on the flow control mode.The replay logic circuit may be further configured to recalculate an adaptive threshold based on the current set of link operation parameters in order to dynamically determine an adaptive threshold for each detected erroneous data link packet. The replay logic circuit may be further configured to dynamically determine the adaptive threshold by processing the set of link operation parameters through a statistical model that includes the set of link operation parameters and corresponding weight coefficients configured to balance the likelihood of successfully recovering the erroneous data link packet with the data link layer latency. The set of link operation parameters may include at least one link connection parameter selected from the current link speed, the number of lanes, and the bit error rate relative to the current link speed and number of lanes. The set of link operation parameters may include at least one link workload parameter selected from the current data link packet workload, the host system historical workload, and the predicted workload. The peripheral interface protocol may support a skip sequence set mode configured to periodically resynchronize the data link layer by introducing idle time during fixed-period transaction layer packet processing, the link set of operating parameters may include counters for fixed-period transaction layer packet processing, and dynamically determining adaptive thresholds depending on the skip sequence set mode may include compensation for predicted idle time based on the counters. The device may be a data storage device comprising a non-volatile storage medium configured to store data for a host system, and a peripheral interface connector configured to connect to a peripheral interface bus for storage communication with the host system using the peripheral interface protocol.

[0007] Another general embodiment is a computer implementation that includes: receiving data link packets from a host system via a peripheral interface using a peripheral interface protocol; determining errors in the data link packets received via the peripheral interface; dynamically determining an adaptive threshold for selecting a replay mode based on a set of link operation parameters; selecting a replay mode selected from a selective replay mode for requesting retransmission of only erroneous data link packets and a sequential replay mode for requesting retransmission of erroneous data link packets and subsequent data link packets transmitted after the erroneous data link packets; and sending a replay request to the host system based on the selected replay mode.

[0008] The implementation may include one or more of the following features: Depending on the selective replay mode, the computer implementation may include storing error-free sequential data link packets received after an error-causing data link packet in a replay buffer memory; sequentially transmitting the transaction layer content from the error-free version of the error-causing data link packet and the error-free sequential data link packets in the replay buffer memory in response to receiving an error-free version of the error-causing data link packet from the host system; and clearing the error-free sequential data link packets from the replay buffer memory. Depending on the sequential replay mode, the computer implementation may include discarding sequential data link packets received after an error-causing data link packet; sequentially transmitting the transaction layer content from the error-free version of the error-causing data link packet and the retransmitted sequential data link packets in response to receiving an error-free version of the error-causing data link packet and a retransmitted sequential data link packet received after the error-free version of the error-causing data link packet. A computer implementation may include periodically determining an adaptive threshold from a set of maximum fill levels based on a set of link operation parameters, wherein the replay buffer memory has a buffer size corresponding to the total number of data link packets that can be stored in the replay buffer memory at one time, the fill level corresponding to the current number of data link packets stored in the replay buffer memory for pending selective replay requests, and the adaptive threshold corresponding to a maximum fill level smaller than the buffer size of the replay buffer memory.A computer implementation may include determining that a flow control mode is active, the peripheral interface protocol supporting a flow control mode configured to change the memory size and format of data link packets, determining the fill level of the replay buffer memory based on the memory size and format of data link packets in the flow control mode, and dynamically determining an adaptive threshold based on the flow control mode. A computer implementation may include recalculating an adaptive threshold based on the current set of link operation parameters in response to each detection of an erroneous data link packet, in order to dynamically determine the adaptive threshold for that erroneous data link packet. A computer implementation may include processing the set of link operation parameters through a statistical model consisting of the set of link operation parameters and corresponding weight coefficients configured to balance the likelihood of successfully recovering an erroneous data link packet with the data link layer latency. The set of link operation parameters may include at least one link connection parameter selected from the current link speed, the number of lanes, and the bit error rate relative to the current link speed and number of lanes. The set of link operation parameters may include at least one link workload parameter selected from the current data link packet workload, the host system historical workload, and the predicted workload. The peripheral interface protocol may support a skip sequence set mode configured to periodically resynchronize the data link layer by introducing idle time between fixed-period transaction layer packet processing, and the link set of operation parameters may include a counter for fixed-period transaction layer packet processing, and depending on the skip sequence set mode, dynamically determining an adaptive threshold may include compensation for the predicted idle time based on the counter.

[0009] A further general embodiment includes a peripheral interface circuit comprising: a data link layer configured to receive data link packets from a host system; a replay buffer memory configured to store received data link packets following an erroneous packet in order to support a selective replay mode; means for determining errors in data link packets received from the host system; means for dynamically determining an adaptive threshold for selecting a replay mode based on a set of link operation parameters; a selective replay mode for requesting retransmission of only erroneous data link packets, based on comparing the fill level of the replay buffer memory with the adaptive threshold; a sequential replay mode for requesting retransmission of erroneous data link packets and subsequent data link packets transmitted after the erroneous data link packets; and means for sending a replay request to the host system based on the selected replay mode.

[0010] Various embodiments advantageously apply teachings of peripheral bus interfaces for computer peripherals to improve the functionality of computer systems including those peripheral devices. These embodiments include operations that overcome, or at least reduce, problems previously encountered in computer systems using peripheral interfaces, and are therefore more reliable and / or efficient than other computing systems. Specifically, the various embodiments disclosed herein include hardware and / or software that have the capability to improve the link latency and reliability of peripheral interfaces, such as by using adaptive thresholds based on link operation parameters to select between selective and sequential replay modes. Therefore, the embodiments disclosed herein provide various improvements to storage networks and / or storage systems.

[0011] Please understand that the language used in this disclosure has been selected primarily for readability and teaching purposes and does not limit the scope of the subject matter disclosed herein. [Brief explanation of the drawing]

[0012] [Figure 1] This schematic diagram illustrates a system for data storage and communication that uses a peripheral interface with an adaptive replay mode selector at the data link layer. [Figure 2] This schematic diagram outlines a system for data communication that focuses on the data link layer transmit / receive components supporting adaptive replay mode selection. [Figure 3] A schematic diagram of a data link layer receiver circuit for managing data packet replay at the peripheral interface of a computing device is shown. [Figure 4] This is a flowchart illustrating an exemplary method for dynamically selecting the replay mode at the data link layer of a PCIe system receiver. [Figure 5] This is a flowchart illustrating an exemplary method for processing packet replay at the data link layer of a PCIe system transmitter. [Figure 6] This is a flowchart illustrating an exemplary method for determining parameters related to the replay buffer within the peripheral interface of a computing device based on the flow control mode. [Figure 7] This is a flowchart illustrating an exemplary method for adjusting the adaptive threshold for packet replay mode based on the skip order set mode. [Figure 8] This is a flowchart illustrating an exemplary method for determining adaptive thresholds for packet replay at peripheral interfaces of computing devices. [Modes for carrying out the invention]

[0013] This disclosure relates to an adaptive replay system for peripheral interfaces. In some examples, the adaptive replay system may dynamically select between selective and sequential replay modes to handle packet errors in data link layer communications. The adaptive replay system may utilize a set of link operation parameters to determine an adaptive threshold for selecting the appropriate replay mode.

[0014] A peripheral interface can connect a device to a host system using a peripheral interface protocol. The peripheral interface protocol may include a data link layer configured to send and receive data link packets. A replay buffer memory may be implemented to support selective replay mode and store received data link packets following an erroneous packet.

[0015] The replay logic circuit may be configured to determine errors in data link packets received via a peripheral interface. The replay logic circuit can dynamically determine an adaptive threshold for selecting a replay mode based on a set of link operation parameters. For example, the link operation parameters may include the current link speed, number of lanes, bit error rate, current workload, and the host system's historical behavior.

[0016] The replay logic circuit can select a replay mode by comparing the fill level of the replay buffer memory with an adaptive threshold. The replay mode may include a selective replay mode that requests retransmission only of erroneous data link packets, and a sequential replay mode that requests retransmission of erroneous data link packets and subsequent data link packets sent after the erroneous data link packets.

[0017] The adaptive replay system can periodically recalculate the adaptive threshold based on the current link operation parameters. This dynamic approach can enable the system to adapt to changing conditions and error patterns, potentially improving overall system performance and reliability. Peripheral interface protocols may support additional features that can influence the adaptive threshold determination. For example, a flow control mode that can change the memory size and format of data link packets may be supported, and / or a skip sequence set mode that can define processing periods and scheduled idle periods to support data link layer resynchronization may be supported. The replay logic can adjust its calculations based on the active flow control mode and / or the skip sequence set mode.

[0018] Adaptive replay systems can be implemented in various devices that utilize peripheral interfaces. For example, the system may be integrated into data storage devices, network interface cards, or other computer peripherals that communicate with a host system using peripheral interface protocols such as PCIe.

[0019] Figure 1 shows an embodiment of an exemplary data storage system 100 having a plurality of data storage devices 120 interconnected by a peripheral interface bus 108. While some exemplary features are shown, various other features are not shown for the sake of brevity and to avoid obscuring relevant aspects of the exemplary embodiments disclosed herein. For this purpose, as a non-limiting example, the data storage system 100 includes one or more data storage devices 120 (sometimes referred to as information storage devices, storage devices, disk drives, or drives). In some embodiments, the storage devices 120 may be configured in servers or storage array blades or similar storage units for use in a data center storage rack or chassis. The storage devices 120 may interface with one or more hosts 102 and provide data storage and retrieval capabilities for or through those host systems. In some embodiments, the storage devices 120 may be configured in a storage hierarchy including storage nodes, storage controllers, and / or other intermediate components between the storage devices 120 and the hosts 102. For example, each storage device may include a peripheral interface circuit and a corresponding peripheral interface connector for connecting to the peripheral interface bus 108, and one or more intermediate systems or components may function as a host system and / or root complex for peripheral interface bus communication.

[0020] In the illustrated embodiment, several storage devices 120 are mounted on a common peripheral interface bus 108 for host communication. For example, the storage devices 120 may include several drives located within a storage array, such as storage devices sharing a common rack, unit, or blade within a data center, or solid-state drives (SSDs) within an entire flash array. In some embodiments, the storage devices 120 may share a backplane network, network switches, and / or other hardware and software components accessed via the peripheral interface bus 108. For example, the storage devices 120 may connect to the peripheral interface bus 108 via a plurality of physical port connections that define physical channels, link channels, and other logical channels for establishing communication with different components and subcomponents to establish a communication channel to the host 102. In some embodiments, the peripheral interface bus 108 may be configured as a storage interface bus and may provide a primary host interface for storage device management and host data transfer.

[0021] In some embodiments, the memory device 120 may be configured to perform peer-to-peer communication between memory devices 120 via the peripheral interface bus 108, with or without the assistance of the host 102. For example, the memory device 120 may be configured for direct memory access using one or more protocols such as non-volatile memory express (NVMe), remote direct memory access (RDMA), NVMe over fabric (NVMeoF), etc. to provide command messaging and data transfer between memory devices using a high-bandwidth storage interface and the storage interface bus 108. In these cases, the memory device 120 may be configured to operate as both an initiator device and an endpoint device for peripheral interface communication, enabling one memory device to function as a host system for initiating communication with another memory device via the peripheral interface bus 108.

[0022] In some embodiments, the data storage device 120 is a solid state drive (SSD) or includes a solid state drive (SSD). Each data storage device 120.1 - 120.n may include a non-volatile memory (NVM) or device controller 130 based on computing resources (processor and memory), and a plurality of NVMs or media devices 140 for data storage (e.g., one or more NVM devices such as one or more flash memory devices). In some embodiments, an individual data storage device 120 of one or more data storage devices may include one or more NVM controllers such as a flash controller or a channel controller (e.g., for a storage device having NVM devices within a plurality of memory channels). In some embodiments, the data storage devices 120 may each be packaged within a housing such as a multi-part sealed housing having a defined form factor and ports and / or connectors for interconnecting with the peripheral interface bus 108.

[0023] In some embodiments, an individual data storage device 120 can include a single media device, but in other embodiments, an individual data storage device 120 includes a plurality of media devices that provide non-volatile storage media for storing host data. In some embodiments, the media device includes NAND-type flash memory or NOR-type flash memory. In some embodiments, the data storage device 120 may include one or more hard disk drives (HDDs). In some embodiments, the data storage device 120 can include a flash memory device, which can include one or more flash memory dies, one or more flash memory packages, one or more flash memory channels, etc. However, in some embodiments, one or more of the data storage devices 120 may have other types of non-volatile data storage media (e.g., phase change random access memory (PCRAM), resistive random access memory (ReRAM), spin transfer torque random access memory (STT-RAM), magnetic random access memory (MRAM), etc.).

[0024] In some embodiments, each storage device 120 includes a device controller 130 that includes one or more processing units (sometimes referred to as a CPU or processor or microprocessor or microcontroller) configured to execute instructions within one or more programs. In some embodiments, the one or more processors are shared by one or more components within, and optionally exceeding, the functionality of the device controller. The media device 140 is typically coupled to the device controller 130 via a connection that conveys commands in addition to data and optionally conveys metadata, error correction information, and / or other information in addition to data values stored in the media device and data values read from the media device 140. The media device 140 can include any number (i.e., one or more) of memory devices including, but not limited to, non-volatile semiconductor memory devices such as flash memory devices.

[0025] In some embodiments, the media device 140 within the storage device 120 is divided into several addressable and individually selectable blocks, sometimes referred to as erase blocks. In some embodiments, an individually selectable block is the smallest erasable unit in the flash memory device. In other words, each block contains the smallest number of memory cells that can be erased simultaneously (i.e., in a single erase operation). Each block is typically further divided into multiple pages and / or word lines, each page or word line being an instance of the smallest individually accessible (readable) portion within the block. In some embodiments (e.g., using some types of flash memory), however, the smallest individually accessible unit of a dataset is a sector or codeword, which is a subunit of a page. That is, a block contains multiple pages, each page contains multiple sectors or codewords, and each sector or codeword is the smallest unit of data for reading data from the flash memory device.

[0026] A data unit can describe any size allocation of data, such as host blocks, data objects, sectors, pages, multiplane pages, erase / programming blocks, and media devices / packages. A storage location can include physical and / or logical locations on the storage device 120 and can be described and / or allocated at different levels of granularity depending on the storage medium, storage device / system configuration, and / or context. For example, a storage location may be allocated with host logical block address (LBA) data unit size and addressing capability for host read / write purposes, but in other contexts it may be managed as a page with storage device addressing managed in the media flash translation layer (FTL). A media segment can include physical storage locations on the storage device 120, and a physical storage location may correspond to one or more logical storage locations. In some embodiments, a media segment can include a contiguous series of physical storage locations, such as adjacent data units on the storage medium, and in the case of a flash memory device, it may correspond to one or more media erase or programming blocks. A logical data group can include multiple logical data units that can be grouped on a logical basis, regardless of their storage location, such as data objects, files, or other logical data structures consisting of multiple host blocks.

[0027] The host system 102, or individual hosts in a system having multiple hosts, may be any suitable computer device, such as a computer, computer server, laptop computer, tablet device, netbook, internet kiosk, personal digital assistant, mobile phone, smartphone, game device, or any other computing device. The host system 102 may also be referred to as a host, client, or client system. In some embodiments, the host system 102 is a server system, such as a server system in a data center, or a storage system, such as a storage array in a data center. In some embodiments, one or more host systems 102 are one or more host devices distinct from a storage controller or storage node that houses multiple storage devices 120. One or more host systems 102 may be configured to store and access data in the multiple storage devices 120.

[0028] The host system 102 may include one or more central processing units (CPUs) 104 for executing computational operations or instructions for accessing the storage device 120 via the peripheral interface bus 108. In some embodiments, the CPU 104 may include a host processor and may be associated with an operational memory 106 for performing both storage operations and storage interface protocols compatible with the peripheral interface bus 108 and the storage device 120. In some embodiments, the memory 106 may include one or more dynamic random access memory (DRAM) devices used by the storage device 120 for storing and transferring commands, management parameters, and / or host data. In some embodiments, the storage device 120 may be configured for direct memory access (DMA), such as using a remote direct memory access (RDMA) protocol, via the peripheral interface bus 108 to access and use host memory buffers allocated to the storage device. The memory 106 may also include an operating system (not shown), firmware, and / or software applications for performing various functions of the host system 102. In some configurations, memory 106 may include a host basic input / output system (BIOS) 106.1 for establishing basic communication between components of the host system 102 during boot-up. In some embodiments, the host BIOS 106.1 may be configured to establish communication to the storage device 120 via the peripheral interface bus 108 during boot-up.

[0029] In some configurations, the peripheral interface bus 108 can interface with a common bus shared by other components of the host system 102, such as the CPU 104 and memory 106. In other configurations, the host system 102 can include interface ports for connecting to the peripheral interface bus 108. For example, the host system 102 can include a peripheral bus interface 108.1 for physically connecting to the peripheral interface bus 108. The host 102, the peripheral interface bus 108, and / or storage interface units can include a root complex 110 for a peripheral interface protocol that manages connections and communications via the peripheral interface bus 108 using a peripheral interface switch 112. For example, the root complex 110 may be configured as a PCIe root complex connecting the CPU and memory subsystems to a PCIe switch fabric consisting of various endpoints that may include storage devices 120, the host system 102, and other devices connected to the peripheral interface bus 108. In some embodiments, the host system 102 may provide processor and memory resources to support the root complex 110. From the perspective of the storage device 120, the peripheral interface bus 108 may be called the host interface bus and provides a host data path between the storage device 120 and the host 102, as well as between other endpoints that can function as host systems to initiate communication via the peripheral interface bus 108.

[0030] System 100 may include a peripheral interface protocol 150 configured to connect a peripheral interface bus 108 to various endpoints connected to the peripheral interface bus 108, such as the host 102 and the storage device 120, for storage communication. The peripheral interface protocol 150 may be based on a transaction layer 152, a data link layer 154, and a physical layer 156. The transaction layer 152 may be responsible for handling higher-level protocol operations such as read and write requests and may manage the flow of transaction layer packets (TLPs) between the host 102 and the storage devices 120.1, 120.2, and 120.n. The transaction layer 152 may handle TLP formation, routing, ordering, and flow control. The data link layer 154 may be responsible for reliable data transfer for the transaction layer 152 via the physical layer 156. For example, the data link layer 154 may assign the contents of transaction layer packets to data link packets for transmission via the physical layer 156. The data link layer 154 may incorporate a replay mode selector 154.1 that can be configured to dynamically select between different replay modes for handling packet errors. The data link layer 154 may implement error detection and correction mechanisms to manage data link packets (DLLPs) and ensure accurate transfer of transaction layer data between the sender and receiver of the data link layer 154 at the corresponding endpoints. The physical layer 156 may be responsible for the actual transmission and reception of data over the physical media of the peripheral interface bus 108. In some cases, the physical layer 156 may include peripheral interface connectors 156.1 for each endpoint connected to the peripheral interface bus 108 and the conductors and switches therein. The physical layer 156 may handle tasks such as signal coding, timing, and electrical specifications for data transmission.

[0031] In some embodiments, the data storage system 100 includes one or more processors, one or more types of memory, a display, and / or other user interface components such as a keyboard, touchscreen display, mouse, trackpad, and / or any number of supplementary devices for adding functionality. In some embodiments, the data storage system 100 does not have a display or other user interface components.

[0032] Referring to Figure 2, a block diagram of system 200 for data communication is shown, focusing particularly on the data link layer of the peripheral interface protocol. System 200 can include a transaction layer 206, a data link layer transmit circuit 202, a data link layer receive circuit 204, and a physical layer 208. Devices implementing the peripheral interface protocol, such as PCIe endpoints, can include both transmit and receive components, but any given transaction lies between the transmit portion of one device and the receive portion of another device. Furthermore, more complex peripheral interface bus connections can include intervening components such as switches and corresponding route complexes to route packets between multiple endpoints.

[0033] The data link layer transmission circuit 202 may process transaction layer content 214, which includes transaction layer packet (TLP) information 216 received from the transaction layer 206. For example, the transaction layer 206 may determine a transaction layer packet for a destination endpoint and generate corresponding transmission data 212 (TLP packet payload) and packet information 210 (TLP packet header and parameters for TLP packet processing). In some configurations, the peripheral interface protocol may implement flow control modes such as FLIT in PCIe. For example, the transaction layer content 214 may be formatted as 236 bytes of TLP information 216. Note that data link packets may not have a one-to-one mapping with transaction layer packets, and depending on the relative size and consistency between transaction layer packets and data link packets, more or fewer transaction layer packets may fit within a data link packet. The data link layer may be based on transmitting sequential data link packets for transaction layer content 214 over the physical layer 208 and receiving and reassembling the sequential data link packets at the data link layer receiver 204 to return the corresponding transaction layer content 252 to the transaction layer 206 at the receiving device. The transaction layer content 214 is then processed via a data link packet (DLP) formatter 218, which may wrap TLP information 216 for transmission according to the link layer protocol. For example, the DLP formatter 218 may add a DLP header and error check data to the TLP information 216 to obtain a data link packet 220. In some configurations, the data link packet 220 may include an additional 6 bytes of DLP parameters 222, such as a 2-byte DLP header and a 4-byte cyclic redundancy check code (CRC) for a 242-byte DLP or FLIT packet. DLP 220 is passed to the physical layer transmitter 224 and transmitted to the destination endpoint via the peripheral interface bus (through switches and route complexes that may be required to reach it).

[0034] In some protocols, the data link layer may generate additional packets to maintain reliable link communication, such as acknowledgments and replay requests, as well as flow control and power management messages. These management packets for the data link layer are sometimes called data link layer packets (DLLPs) 226. A DLLP 226 may contain DLLP parameter data 228, such as message type and information. These DLPs 226 may also receive DLP parameters 222 (header and CRC) from a DLP formatter 218 before being transmitted via a physical layer transmitter 224. A DLLP 226 containing flow control information may receive its flow control information 238 from a corresponding data link layer receiver 204, which may include a receiver and / or an intermediate receiver such as a switch at the destination endpoint.

[0035] In some configurations, the data link layer transmit circuit 202 may include a transmit replay or retry buffer 232 for storing data that may require retransmission. The transmit retry buffer 232 may be configured to store a variable number of transaction layer packets for replay in response to replay requests from the data link layer receive circuit 204, depending on the current operating conditions and protocol settings. In some configurations, the transmit retry buffer 232 may be sized and managed to store all transaction layer packet information 216 for TLPs that have been transmitted but not acknowledged via the corresponding DLLP. The data link layer transmit circuit 202 can use a multiplexer 230 with route selection logic to select between a new TLP received from the transaction layer 206 and a replay of a TLP from the transmit retry buffer 232. For example, the multiplexer 230 may select a replay packet from the transmit retry buffer 232 using first-in, first-out (FIFO) selection when a replay request is pending to help clear the old TLP from the transmit replay buffer 232. In some configurations, the storage interface protocol may also support no-operation (NOP) data packets from the transaction layer 206 as no-operation data content 234 and corresponding TLP information 236. These TLPs can be treated like regular TLPs, but given different priorities are assigned via the multiplexer 230. In some cases, NOP data packets may be sent when there are no other packets available to be sent to help maintain link connectivity and operational parameters.

[0036] On the receiving side, the data link layer receiving circuit 204 can process incoming data link packets 242 received via the physical layer receiver 240 from a transmitter in another device, such as a host system or another endpoint. The data link packet 242 may contain TLP information 216 and DLP parameters 222 corresponding to TLP information 244 and DLP parameters 246 transmitted by the sender, and may be subject to errors caused by any transmission. The DLP processor 248 can parse the DLP 242 and use the DLP parameters 246 for packet identification and error detection. For example, the DLP header may identify the DLP packet and its contents, and the CRC may be used to determine whether one or more errors are present in the data link packet. Data link packets that do not contain errors have the removed DLP parameters and may be stored in the receive replay or retry buffer 250. For example, the TLP or FLIT data corresponding to the original 236 bytes in the transaction layer content 214 and TLP information 216 may be stored in the receive retry buffer 250 as the corresponding DLP unit. Errored data link packets can generate replay requests that are returned to the transmitter in the DLLP. The replay mode selector 260 can dynamically and adaptively select whether selective replay mode or sequential replay mode is used for the replay request, as will be further illustrated with reference to Figure 3. Errored data link packets may be discarded, and the processing of preceding and / or succeeding sequential packets related to the same TLP may be determined by the selected replay mode.

[0037] Error-free packets are received and aggregated in the receive retry buffer 250. Transaction layer content 252 corresponding to TLP information 244 can be reassembled into a TLP and returned to the transaction layer 206 as received data 254 (transaction packet payload) and packet information (transaction packet parameters) for further processing. Once the TLP is complete and returned to the transaction layer 206, an acknowledgment for that TLP is sent to the transmitter using a DLLP, allowing both the data link transmit circuit 202 and the data link layer receive circuit 204 to clear the corresponding transaction layer content from their respective transmit retry buffers 232 and receive retry buffers 250. The DLP processor 248 can also update flow control information 238 and send the corresponding flow control DLLP to the transmitter.

[0038] Figure 3 schematically shows a selected module of the data link layer receiver circuit 300 configured to dynamically adjust adaptive thresholds for selection between replay modes. For example, the data link layer receiver circuit 300 can embody some of the data link layer functions described with respect to Figures 1 and 2 for peripheral interfaces such as a PCIe interface for a data storage device to communicate with a host system. Endpoint devices including the receiver circuit 300 may also include other elements from Figures 1 and 2 to support broader functionality for transmitting, receiving, and managing flow control and other features of peripheral interfaces and the peripheral interface protocols it implements. For example, the receiver circuit 300 may also include components such as an interface to a physical layer receiver, connections to transmitting circuit components such as a DLP processor, TLP logic for reassembling TLP packets from error-free data link packets in a receive replay buffer 350 or another receive buffer, flow control logic, and a DLLP generator for transmitting replay requests and acknowledgments.

[0039] The receiving circuit 300 may include at least one processor 312, at least one memory 314, and a bus 310 interconnecting the other components of the receiving circuit 300. The bus 310 may include one or more conductors that enable communication between the components of the receiving circuit 300 and / or other components of the peripheral interface. The processor 312 may include any type of processor or microprocessor that interprets and executes instructions or operations, and may include multiple processors or processor cores configured to operate individually or in combination. The memory 314 may include random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by the processor 312, and / or read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by the processor 312. The memory 314 may include one or more memory devices, and may include memory devices of different types and configurations. In some configurations, some or all of the other components in the receiving circuit 300 may be instantiated in the memory 314 for execution by the processor 312. For example, the parameter register 360 and the replay buffer 350 may be embodied in data structures allocated in memory 314, and the replay logic 320 and the data link state machine 380 may be instantiated by a set of instructions and data structures in memory 314 executed by the processor 312 to perform the described functions. In some configurations, one or more components may be instantiated in hardware circuits having their own logic and memory circuits for performing the described functions, and the processor 312 and memory 314 may provide coordination and management of the functions performed by those hardware circuits via the bus 310.

[0040] The receiving circuit 300 may include a replay logic circuit 320 comprising logic for processing errors detected in the received data link packets, managing the capacity of the replay buffer 350, and dynamically selecting between replay modes in response to detected packet errors. The replay logic circuit 320 may include an error detector 322 for identifying errors in the received data packets. For example, the error detector 322 may extract a CRC value from each DLP and perform a CRC check on the received DLP to determine whether an error caused by transmission is present in the DLP. The error detector 322 may include CRC processing logic that determines, based on the payload size (FLIT data) and the CRC code added by the transmitter, whether the packet contains an error and whether replay is required to successfully return the TLP data in that DLP.

[0041] The receiver circuit 300 may include a replay mode selector 324 within a replay logic circuit 320 configured to determine whether to use sequential replay mode 326 or selective replay mode 328 based on various factors. In some configurations, the receiver circuit may include a static threshold for selecting between sequential replay mode 326 and selective replay mode 328. For example, the receiver circuit 300 may default to selective replay mode 326 until the fill level of the replay buffer 350 meets a static threshold for triggering sequential replay mode 328. While the use of a static threshold may allow the receiver to utilize both modes, it does not optimize their use due to the dynamic nature of the data link connection and the changing workload of the peripheral interface. Therefore, the replay mode selector 324 may be configured with a dynamic model for determining an adaptive threshold that is periodically adjusted based on a set of link operation parameters. For example, the statistical engine 330 may implement one or more statistical models for changing the adaptive threshold in response to changing link operation conditions and current and / or predicted workloads. In some configurations, the statistics engine 330 can calculate a new adaptive threshold each time an erroneous packet is detected by the error detector 322. The replay logic circuit 320 can dynamically determine the adaptive threshold for each erroneous data link packet by recalculating the adaptive threshold 330.1 based on the current set of link operation parameters. For example, whenever the error detector 322 identifies a packet error, the statistics engine 330 can analyze the current link speed 364, number of lanes 366, bit error rate 368, and other relevant parameters to recalculate an appropriate adaptive threshold 330.1 for that particular error instance. In other configurations, threshold adaptation can be triggered based on a periodic schedule, such as time, number of packets, skip sequence set (SKPOS) resynchronization, or another operating period. Other trigger conditions are also possible, such as based on a bit error rate (BER) threshold, a link speed change threshold, etc.

[0042] Sequential replay mode 326 may include a conventional link replay mode configured to resume the sequence of DLPs for current and future TLPs without storing TLP data from error-free DLPs in the replay buffer 350. This mode may be used when the replay buffer 350 is full or otherwise unavailable, and may be used selectively to allow clearing of previously requested packet errors using selective replay mode 328. Sequential replay mode may be triggered in response to error detection by the error detector 322, such as when a packet that has failed integrity check using an error detection mechanism such as CRC is received. Error notification may be provided to the transmitter by sending a replay request indicating that the entire TLP (which may contain multiple DLPs) should be retransmitted. Upon receiving the replay request, the transmitting device can find the TLP in its transmit replay buffer and generate DLPs to retransmit the TLP sequence. When the receiving device receives the erroneous packet (and other retransmitted DLPs to complete the TLP), the TLP may be reassembled and returned to the transaction layer, while the receiver sends an acknowledgment to the transmitter.

[0043] Selective replay mode 328 may include selective replay of only erroneous DLPs to avoid retransmission of other DLPs in the TLP sequence. Selective replay mode may be based on storing error-free DLPs in the replay buffer 350 to enable the selection and rearrangement of DLPs to complete the TLP. Selective replay mode 328 may be initiated based on selection by the error detector 322 and the replay mode selector 324. A replay request sent to the transmitter indicates a selective replay mode and may identify erroneous DLP packets. The transmitting device retransmits only the requested packets. The receiving circuit 300 continues to receive and process good packets received after erroneous packets in the DLP sequence (and the corresponding TLP sequence). Selective replay mode 328 reorders the replay buffer 350 and uses it as a buffer to store good packets until the erroneous packets are retransmitted by the transmitter and received without errors. The receiving device acknowledges the completion resulting from the retransmitted packet and / or the TLP passed to the transaction layer.

[0044] The statistical engine 330 within the replay logic circuit 320 may be configured using one or more statistical models to determine the adaptive threshold 330.1. The adaptive threshold 330.1 may include a threshold that, when met, triggers a change in the replay mode from selective replay mode 328 to sequential replay mode 326, and can trigger a return to selective replay mode when the threshold is no longer met. For example, the adaptive threshold 330.1 may correspond to the fill level of the replay buffer 350, more specifically, the maximum fill level, above which the sequential replay mode is selected to reduce the pressure on the replay buffer 350. The adaptive threshold 330.1 may vary within a range of maximum fill level values ​​smaller than the total capacity of the replay buffer 350. The adaptive threshold 330.1 may correspond to a maximum fill level smaller than the buffer size of the replay buffer memory 350. For example, for a given operating period, the adaptive threshold 330.1 may be set to 80% of the total buffer size 362 and then adapt to allow a variable amount of buffer headroom as the link operating parameters change. In some configurations, the fill level and adaptive threshold 330.1 may be incremented in steps corresponding to the packet size of error-free packets held in the replay buffer 350. For example, the FLIT size and the number of packets may be used to represent the capacity, fill level, and / or adaptive threshold 330.1.

[0045] The statistical model used by the statistical engine 330 may be based on a set of link operation parameters 330.2. For example, the statistical engine 330 may be configured for a set of link operation parameters representing link state and workload, which may be associated with each other to determine that thresholds should be fitted to changing conditions. In some configurations, the statistical engine 330 can determine, request, or access operation parameter values ​​for operation parameters 330.2 from a parameter register 360. The parameter register 360 may include combinations of link connection parameters such as the current link speed 364, number of lanes 366, and bit error rate 368, and link workload parameters such as the current workload 370 and host workload history 372. In a simple statistical model, each selected operation parameter may be included as a term in the model, and each term may include associated operators and weight coefficients 330.3 to represent the relationships and trade-offs between different parameters. In some configurations, the statistical engine 330 may be configured to balance a completion target 330.4 for determining the likelihood that an erroneous packet (and / or the TLP or TLP sequence to which it contributes) will be successfully replayed and received without errors, and a latency target 330.5 for determining changes in data link layer latency based on changes in replay mode. In some configurations, the completion target 330.4 may be based on a completion risk threshold, such as a 90% probability of recovering the erroneous packet in the next attempt, and the latency target 330.5 may be based on an acceptable latency change threshold, such as a 5% increase in link latency. FLIT compensation 330.6 may include a statistical model or factor within the statistical engine 330 that responds to the activation of flow control mode, as well as the accompanying changes in packet size and format and their impact on the replay buffer 350, and the reorder or completion logic for the corresponding TLP.SKPOS compensation 330.7 may include factors in a statistical model or logic in the statistical engine 330 that respond to the activation of skip sequence set modes and associated processing durations and idle times, providing predictable interruptions in DLP packet processing and supporting catch-up to the error backlog. The statistical engine 330 may be configured to compensate for any number or combination of operating parameters and / or operating modes that affect link operation.

[0046] The replay logic circuit 320 may include a replay request interface 332 for initiating packet retransmission via a replay request to the transmitting device, such as by initiating a DLLP message that identifies the erroneous packet and the requested replay mode. The replay logic circuit 320 may include a reorder logic 334 for reassembling DLP and TLP sequences in response to error-free packets stored in the replay buffer 350 and / or received in response to a replay request, and returning the completed TLP to the transaction layer in accordance with the error-free version of the erroneous packet that was the subject of the replay request. The replay logic circuit 320 may include an acknowledgment interface 336 for confirming the success of the transmission, such as by initiating a DLLP message indicating that the DLP packet was received without error and / or that the TLP was completed without error. The replay logic circuit 320 may include a buffer monitor 338 for tracking the status of the replay buffer 350 and, more specifically, determining the current fill level based on error-free packets stored in the replay buffer 350 waiting for the completion of those sequences and the return of the corresponding TLPs to the transaction layer. The replay logic circuit 320 may be configured to select a replay mode based on comparing the fill level of the replay buffer memory 350 with an adaptive threshold. For example, a buffer monitor 338 may track the current fill level of the replay buffer 350, and a replay mode selector 324 may compare this fill level with an adaptive threshold 330.1 to determine whether to use sequential replay mode 326 or selective replay mode 328.

[0047] The replay buffer memory 350 may include one or more memory devices configured to store data link packets 352.1, 352.2, 352.3, and up to 352.n. The replay buffer memory 350 may be configured as a FIFO buffer having a total capacity based on the allocated device memory size and the packet size of packet 352. In some configurations, the replay buffer 350 may be indexed to allow for the reordering of packets based on an intended sequential order (such as determined at the transmitter) and represented in the header parameters of the DLP and / or TLP mapping data. The replay buffer memory 350 may be configured to store error-free received data link layer packets following an error-causing packet, supporting selective replay mode and sequential packet reassembly when an error-causing, corrected packet is received. In some configurations, the replay buffer memory 350 may have a buffer size corresponding to the total number of data link packets that can be stored in the replay buffer memory 350 at one time. For example, the buffer size 362 stored in the parameter register 360 may indicate the maximum capacity of the replay buffer memory 350. The fill level of the replay buffer memory 350 may correspond to the current number of data link packets stored in the replay buffer memory 350 for pending selective replay requests. The buffer monitor 338 can track this fill level in real time as packets are added to and removed from the replay buffer memory 350.

[0048] The parameter register 360 may include one or more memory devices configured to store various link operation parameters. Exemplary link operation parameters that may be included in the parameter register 360 include the replay buffer size 362, the current link speed 364, the number of lanes 366, the BER 368 (relative to the current link speed and number of lanes), the current data link packet workload 370, the host system history 372 (historical workload for a particular host system / transmitter), the FLIT mode indicator 374, and the SKPOS counter 376. Some or all of these parameters may be used by the replay logic circuit 320 to determine an adaptive threshold 330.1 and to dynamically select between replay modes. For example, the statistics engine 330 may use a set of link operation parameters, including link connection parameters selected from the link speed 364, lanes 366, and BER 368, to model the link latency and error potential. In another example, the statistical engine 330 may use a set of link operation parameters, including the workload 370, link workload parameters selected from the host history 372, and / or a predicted workload amount based on a predictive model that extrapolates the current workload, host history, and changes in the current workload over future periods. In some configurations, one or more of the parameter values ​​in the parameter register 360 may be determined by the data link state machine 380. For example, the data link state machine 380 may monitor the link speed 364, lane connection 366, BER 368, current workload, and various modes (including FLIT and SKPOS modes) to maintain link operation at a desired performance level. In some configurations, the parameter register 360 may include a set of memory registers associated with the data link state machine 380 and / or other functions in or connected to the receiving circuit 300, which may be updated and accessed to provide the current values ​​of each parameter.

[0049] In some configurations, the replay logic circuit 320 may be configured to determine if a flow control mode is active. For example, the FLIT mode 374 parameter in the parameter register 360 may indicate whether a flow control mode, such as FLIT (Flow Control Unit), is currently in use. When the FLIT mode is active, the replay logic circuit 320 may adjust its calculation and decision-making processes to take into account the different packet sizes and formats used in the FLIT mode. The statistical engine 330 can use various compensation mechanisms to fine-tune the adaptive threshold 330.1. For example, the FLIT compensation 330.6 may adjust its calculations to take into account specific characteristics of the FLIT mode when it is active.

[0050] Similarly, the SKPOS compensation 330.7 can modify the threshold based on the frequency and timing of skip sequence sets indicated by the SKPOS counter 376. The replay mode selector 324 may consider the distance from the SKPOS time window or the current packet processing window. In some configurations, the SKP OS is a periodic marker inserted into the data stream to maintain synchronization between the transmitter and receiver, introducing idle time to allow for data link layer resynchronization. The parameter register 360 may include the SKPOS counter 376 for tracking the timing and frequency of these SKP OS occurrences. The SKPOS counter 376 may provide information about the timing of skip sequence sets that may affect the packet transmission pattern. The replay mode selector 324 can use this information to adjust its replay mode selection strategy. The SKPOS compensation 330.7 can use the value from the SKPOS counter 376 to adjust the adaptive threshold 330.1 or influence the replay mode selection. For example, if the SKPOS counter 376 indicates that an SKP OS is imminent, the replay logic circuit 320 can adjust its decision-making process to avoid initiating a sequential replay operation that might conflict with the next SKP OS. If a packet error occurs near the next SKP OS window, the replay mode selector 324 can increase the adaptive threshold 330.1 and, depending on the idle time provided by the SKP OS resynchronization, receive a packet retransmission before the next processing window begins, rather than interrupting the sequential replay.

[0051] Figure 4 shows a flowchart of method 400 for dynamically selecting the replay mode in a PCIe system. Method 400 can be performed by a receiving circuit in a data storage device or other peripheral device connected to the host system via a PCIe interface. Method 400 can result in efficient handling of packet errors while optimizing buffer usage and system performance. Generally, method 400 can dynamically select between selective replay mode and sequential replay mode based on the replay buffer fill level and adaptive threshold.

[0052] In block 410, packets may be received from the transmitter. For example, a receiving circuit may receive data link packets from a host system via a PCIe interface.

[0053] In block 412, errors in the received packet can be determined. For example, the receiving circuit may perform a cyclic redundancy check on the received packet to detect any transmission errors.

[0054] In block 414, an adaptive threshold can be determined. For example, the receiving circuit can calculate the threshold based on current link operation parameters such as link speed, number of lanes, and bit error rate.

[0055] In block 416, the fill level of the replay buffer can be determined. For example, the receiving circuit can check the memory allocation in its replay buffer memory, which is used to store good packets waiting for the replay of erroneous packets during selective replay.

[0056] In block 418, a decision may be made based on whether the fill level meets a threshold. For example, the receiving circuit may compare the current replay buffer fill level with a calculated adaptive threshold.

[0057] In block 420, if the fill level does not meet the threshold, selective replay mode may be selected. For example, the receiving circuit may choose to use selective replay if sufficient buffer space is available.

[0058] In block 422, a replay request for an erroneous packet may be sent. For example, the receiving circuit may send a data link layer packet (DLLP) to the host system to request retransmission of a specific erroneous packet and specify a selective replay mode.

[0059] In block 424, sequential packets can be stored in a replay buffer. For example, a receiving circuit can continue to receive and buffer subsequent error-free packets while waiting for the retransmission of an error-causing packet.

[0060] In block 426, the replay packet may be received without errors. For example, the receiving circuit can receive the packet retransmitted from the host system, verify its integrity, and confirm an error-free version.

[0061] In block 428, transaction layer content from replay packets and stored packets may be sent to the transaction layer. For example, the receiving circuit may forward one or more transaction layer packets corresponding to a corrected packet sequence, including error-free versions of erroneous packets, to a higher protocol layer for further processing.

[0062] In block 430, transmitted packets may be cleared from the replay buffer. For example, a receiving circuit may remove successfully processed packets from its buffer to make space.

[0063] In block 440, sequential replay mode may be selected if the fill level satisfies a threshold that may include being equal to or exceeding an adaptive threshold. For example, the receiving circuit may choose to use sequential replay if the buffer space is limited and the operating conditions suggest that selective replay is unlikely to succeed or be more efficient.

[0064] In block 442, a replay request may be sent for an erroneous packet. For example, the receiving circuit may send a DLLP to the host system requesting a retransmission starting with the erroneous packet.

[0065] In block 444, subsequent packets following an error may be discarded. For example, the receiving circuit may ignore any subsequent packets received after detecting an error.

[0066] In block 446, the replay packet may be received without error. For example, a receiving circuit can accept the packet retransmitted from the host system and verify its integrity.

[0067] In block 448, transaction layer content from the replay packet may be sent to the transaction layer. For example, the receiving circuit may forward the transaction layer packet corresponding to the corrected packet to a higher protocol layer for further processing.

[0068] In block 450, transaction layer content from sequential packets may be received and transmitted to the transaction layer. For example, a receiving circuit may receive and forward the remaining retransmitted packet sequence to complete one or more transaction layer packets, including the content from the corrected packet and / or the subsequent packets.

[0069] Figure 5 shows a flowchart of Method 500 for handling packet replay at the data link layer of a PCIe system transmitter. Method 500 can be implemented by a transmit circuit in a data storage device or other peripheral device connected to a host system via a PCIe interface. Method 500 can result in efficient handling of packet errors by dynamically selecting between selective replay mode and sequential replay mode. In general, Method 500 can provide a structured approach to error handling in data transmission and allow flexibility in selecting the most appropriate replay mode based on the specific circumstances of the error.

[0070] In block 510, packets may be sent to the receiver. For example, the transmitting circuit may send data link packets to the host system via the PCIe interface.

[0071] In block 512, a replay request may be received. For example, the transmitting circuit may receive a data link layer packet (DLLP) from the host system indicating that an error occurred in the transmission of a previously transmitted packet.

[0072] In block 514, a specific packet with an error may be determined. For example, the transmitting circuit may analyze the replay request and identify which packet needs to be retransmitted based on the data link packet identifier from the header of the erroneous packet.

[0073] In block 516, the replay mode to be used may be determined. For example, the transmitting circuit may determine the replay mode parameter value from a replay request set by the receiver that detected the error.

[0074] In block 518, a determination may be made as to whether selective replay should be used. For example, the transmitting circuit may determine whether the value of the replay mode parameter indicates selective replay mode or sequential replay mode.

[0075] If selective replay is indicated in block 520, the packet sequence may be interrupted. For example, the transmitting circuit may pause the current transmission sequence to prepare to retransmit a particular packet that contains an error.

[0076] In block 522, the erroneous packet may be deselected from the sequence and retransmitted. For example, the transmitting circuit may remove the erroneous packet from its transmit buffer and transmit it immediately, regardless of its original position in the sequence.

[0077] In block 524, the packet sequence may be resumed from the point of interruption. For example, after a retransmitted packet, the transmitting circuit may return to transmitting a new packet in the ongoing sequence without repeating the packets that followed the initially retransmitted packet.

[0078] If selective replay is not indicated in block 530, the current packet sequence may be terminated. For example, the transmitting circuit may stop the ongoing transmission and prepare to retransmit all packets from the point of the error.

[0079] In block 532, the erroneous packet may be retransmitted. For example, the transmitting circuit can remove the erroneous packet from its transmit buffer and begin retransmission from that point.

[0080] In block 534, the packet sequence can be resumed from the packet that had the error. For example, the transmitting circuit can continue transmitting all subsequent packets in the sequence and effectively retransmit a portion of the data stream that continues forward, starting from the packet with the error.

[0081] Figure 6 shows a flowchart of Method 600 for determining parameters related to a replay buffer in a peripheral interface of a computing device based on the flow control mode. Method 600 can be performed by a replay logic circuit in a data storage device or other peripheral device connected to the host system via the peripheral interface. Method 600 can result in accurate determination of replay buffer parameters that take into account variations in packet size and format due to the flow control mode. Generally, Method 600 can dynamically adjust the replay buffer parameters to optimize error handling and data transmission efficiency in different flow control scenarios.

[0082] In block 610, the buffer size of the replay buffer can be determined. For example, the replay logic circuit can access the parameter register to retrieve the total memory capacity allocated to the replay buffer.

[0083] In block 612, the flow control mode can be determined. For example, the replay logic circuit may check the current status of the peripheral interface protocol to determine whether a particular flow control mode, such as FLIT (flow control unit) mode, is active.

[0084] In block 614, the packet size and format, as well as the replay buffer packet capacity, may be determined based on the flow control mode. For example, if FLIT mode is active, the replay logic circuit may calculate the new packet size and format according to the FLIT specification, and then determine how many of these packets can be stored in the replay buffer.

[0085] In block 616, the fill level of the replay buffer may be determined based on the packet size and format of the current flow control mode. For example, the replay logic circuit can calculate the current occupancy of the buffer by counting the number of packets currently stored in the replay buffer and multiplying it by the packet size.

[0086] Figure 7 shows a flowchart of Method 700 for adjusting the adaptive threshold for packet replay mode based on skip sequence set mode. Method 700 can be implemented by replay logic circuitry in a data storage device or other peripheral device connected to the host system via a peripheral interface. Method 700 can result in optimized adaptive threshold adjustment that takes into account periodic resynchronization events at the data link layer. Generally, Method 700 can dynamically adjust the adaptive threshold based on processing duration and predicted idle time to improve the efficiency of packet replay decisions.

[0087] In block 710, the skip sequence set mode can be determined. For example, the replay logic circuit may check the current status of the peripheral interface protocol to determine whether the skip sequence set mode is active.

[0088] In block 712, the processing period and idle time for resynchronization may be determined. For example, the replay logic circuit may retrieve configured values ​​for processing transaction layer packets for a fixed period and scheduled or predicted idle time between these periods.

[0089] In block 714, packets may be received from the transmitter. For example, the replay logic circuit can receive incoming data link packets from the host system via a peripheral interface.

[0090] In block 716, a counter for the processing period may be incremented. For example, the replay logic circuit may update an internal counter that tracks progress within the current processing period.

[0091] In block 718, the current count to idle can be determined. For example, the replay logic circuit may calculate the remaining time or packet count before the next scheduled idle period based on the current counter value and a known processing duration.

[0092] In block 720, the adaptive threshold may be adjusted based on the current count to idle. For example, the replay logic circuit may modify the adaptive threshold for selection between replay modes, potentially increasing the threshold as the system approaches an idle period to take advantage of the next resynchronization opportunity.

[0093] Figure 8 shows a flowchart of Method 800 for determining an adaptive threshold for packet replay in a peripheral interface of a computing device. Method 800 can be implemented by a replay logic circuit in a data storage device or other peripheral device connected to the host system via the peripheral interface. Method 800 can result in a dynamically adjusted adaptive threshold that optimizes the selection between replay modes based on current system conditions. Generally, Method 800 can analyze various system parameters and historical data to calculate an adaptive threshold that balances efficient error handling with system performance.

[0094] In block 810, a statistical model may be determined. For example, the replay logic circuit may select or configure a statistical model that incorporates relevant system parameters and their relationships for threshold calculation.

[0095] In block 812, a completion target may be determined. For example, the replay logic circuit may set a target percentage for successful packet transmission completion based on system requirements and historical performance data.

[0096] In block 814, a latency target may be determined. For example, the replay logic circuit may establish a maximum allowable latency for packet retransmission in order to maintain overall system responsiveness.

[0097] In block 816, parameter types and weight coefficients can be determined. For example, the replay logic circuit can identify key system parameters such as link speed and bit error rate and assign appropriate weight coefficients to each based on their relative importance in the statistical model.

[0098] In block 820, packet errors may be detected. For example, the replay logic circuit may identify a cyclic redundancy check (CRC) failure in a received data link layer packet.

[0099] In block 822, the current link speed can be determined. For example, the replay logic circuit can query the physical layer to obtain the current operating speed of the peripheral interface link.

[0100] In block 824, the number of lanes can be determined. For example, the replay logic circuit may examine the current link configuration to identify the number of active lanes in the peripheral interface connection.

[0101] In block 826, the bit error rate can be determined. For example, the replay logic circuit can calculate or retrieve the current bit error rate based on recent transmission statistics.

[0102] In block 828, the packet workload can be determined. For example, the replay logic circuit can measure the current rate of packet transmission and reception on the peripheral interface.

[0103] In block 830, the host historical workload amount can be determined. For example, the replay logic circuit can access stored data about the past workload patterns of the connected host system.

[0104] In block 832, the predicted workload can be determined. For example, the replay logic circuit can use machine learning algorithms or statistical analysis to predict the expected workload based on current trends and historical data.

[0105] In block 834, adaptive thresholds can be calculated based on the collected parameters and factors. For example, the replay logic circuit can input all determined values ​​into a statistical model to calculate new adaptive thresholds for selecting between replay modes, in particular, for packets in which errors have been detected.

[0106] While at least one exemplary embodiment is presented in the preceding detailed description of the Art, it should be understood that a vast number of variations may exist. It should also be understood that one or more exemplary embodiments are examples and are not intended in any way to limit the scope, applicability, or configuration of the Art. Rather, the preceding detailed description provides a convenient roadmap for implementing exemplary embodiments of the Art to those skilled in the art, and it should be understood that various modifications can be made to the function and / or arrangement of the elements described in the exemplary embodiments without departing from the scope of the Art, as described in the appended claims and their legal equivalents.

[0107] As will be understood by those skilled in the art, various aspects of this technology may be embodied as systems, methods, or computer program products. Accordingly, some aspects of this technology may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or combinations of hardware and software embodiments, all of which can be generally referred to herein as circuits, modules, systems, and / or networks. Furthermore, various aspects of this technology may take the form of computer program products embodied in one or more computer-readable media, including computer-readable program code embodied thereon.

[0108] Any combination of one or more computer-readable media can be used. A computer-readable media may be a computer-readable signal medium or a physical computer-readable storage medium. A physical computer-readable storage medium may, but is not limited to, electronic, magnetic, optical, crystalline, polymer, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Non-limiting examples of physical computer-readable storage media may include, but are not limited to, electrical connections involving one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical processors, magnetic processors, or any suitable combination thereof. In the context of this disclosure, a computer-readable storage medium may be any tangible medium capable of containing and / or storing programs or data for use by, or in connection with, an instruction execution system, apparatus, or device.

[0109] Computer code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, fiber optic cable, radio frequency (RF), or any preferred combination thereof. Computer code for performing the operation of aspects of this technology may be written in any static language, such as the C programming language or other similar programming languages. The computer code may run entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the latter scenario, the remote computing device may be connected to the user's computing device via any type of network or communication system, including but not limited to a local area network (LAN) or wide area network (WAN), or a converged network, or it may be connected to an external computer (for example, via the Internet using an Internet service provider).

[0110] Various aspects of this technology may be described above with reference to flowcharts and / or block diagrams of methods, apparatus, systems, and computer program products. It will be understood that each block in the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing device (processor) of a general-purpose computer, a dedicated computer, or other programmable data processing device, which can generate a machine such that instructions that can be executed via the processing device or other programmable data processing device create means for performing the operations / functions specified in the blocks of the flowchart and / or block diagram.

[0111] Some computer program instructions may also be stored in computer-readable media that can instruct a computer, other programmable data processing device, or other device to operate in a particular way, and instructions stored in computer-readable media may generate a product that includes instructions to perform the operations / functions specified in the blocks of a flowchart and / or block diagram. Some computer program instructions may also be loaded onto a computing device, other programmable data processing device, or other device to cause a series of operation steps to be executed on the computing device, other programmable device, or other device, thereby generating computer-executed processes such that instructions executed by a computer or other programmable device provide one or more processes to perform the operations / functions specified in the blocks of a flowchart and / or block diagram.

[0112] The flowcharts and / or block diagrams in the above figures may illustrate the architecture, functionality, and / or operation of possible implementations of devices, systems, methods, and / or computer program products according to various aspects of the present technology. In this regard, blocks in a flowchart or block diagram may represent a module, segment, or portion of code that may contain one or more executable instructions for implementing one or more specified logical functions. Note that in some alternative embodiments, some functions described in a block may be performed out of the order shown in the diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or blocks may sometimes be executed in reverse order depending on the operations they contain. Note that blocks in block diagrams and / or flowcharts, or combinations of blocks in block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system, or a combination of dedicated hardware and computer instructions, that can perform one or more specified operations or functions.

[0113] Although one or more embodiments of this technology have been illustrated and described in detail, those skilled in the art will understand that modifications and / or adaptations to various embodiments can be made without departing from the scope of this technology as set forth in the following claims.

Claims

1. A device having a peripheral interface configured to connect to a host system using a peripheral interface protocol, The aforementioned peripheral interface protocol includes a data link layer configured to send and receive data link packets, The aforementioned peripheral interface is To support selective replay mode, a replay buffer memory is configured to store received data link packets following error packets, The replay logic circuit comprises, Determine the error in the data link packet received via the aforementioned peripheral interface. Dynamically determine an adaptive threshold for selecting the replay mode based on a set of link operation parameters. Based on comparing the fill level of the replay buffer memory with the adaptive threshold, The selective replay mode for requesting retransmission of only the data link packets containing the aforementioned error, A replay mode is selected from the following: a sequential replay mode for requesting the retransmission of the erroneous data link packet and subsequent data link packets transmitted after the erroneous data link packet. A device configured to send a replay request to the host system based on the selected replay mode.

2. The aforementioned replay logic circuit is Depending on the selective replay mode, The error-free sequential data link packets received after the erroneous data link packet are stored in the replay buffer memory. Upon receiving an error-free version of the erroneous data link packet from the host system, the replay buffer memory sequentially transmits the error-free version of the erroneous data link packet and the transaction layer content from the error-free sequential data link packets. The error-free sequential data link packets are cleared from the replay buffer memory. Depending on the sequential replay mode, The data link packets received sequentially after the erroneous data link packet are discarded. The device according to claim 1, further configured to sequentially transmit transaction layer content from the error-free version of the error-containing data link packet and the retransmitted sequential data link packet in response to receiving the error-free version of the error-containing data link packet and a retransmitted sequential data link packet received after the error-free version of the error-containing data link packet.

3. The replay buffer memory has a buffer size corresponding to the total number of data link packets that can be stored in the replay buffer memory at one time. The fill level corresponds to the current number of data link packets stored in the replay buffer memory for pending selective replay requests. The adaptive threshold corresponds to a maximum fill level smaller than the buffer size of the replay buffer memory. The replay logic circuit is further configured to periodically determine the adaptive threshold from a plurality of maximum fill levels based on the set of link operation parameters. The device according to claim 1.

4. The peripheral interface protocol supports a flow control mode configured to change the memory size and format of the data link packets. The aforementioned replay logic circuit is It is determined that the flow control mode is active, Based on the memory size and format of the data link packet in the flow control mode, the fill level of the replay buffer memory is determined. The system is further configured to dynamically determine the adaptive threshold based on the flow control mode. The device according to claim 3.

5. The device according to claim 1, wherein the replay logic circuit is further configured to recalculate the adaptive threshold based on the current set of link operation parameters in order to dynamically determine the adaptive threshold for each erroneous data link packet detected.

6. The device according to claim 1, wherein the replay logic circuit is further configured to dynamically determine the adaptive threshold by processing the set of link operation parameters via a statistical model that includes the set of link operation parameters and corresponding weight coefficients configured to balance the likelihood of successfully recovering the erroneous data link packet with data link layer latency.

7. The set of link operation parameters is: Current link speed and The number of lanes, The device according to claim 1, comprising at least one link connection parameter selected from the current link speed and bit error rate with respect to the number of lanes.

8. The set of link operation parameters is: Current data link packet workload volume and Host system historical workload volume and The device according to claim 1, comprising a predicted workload volume and at least one link workload parameter selected from.

9. The peripheral interface protocol supports a skip sequence set mode configured to periodically resynchronize the data link layer by introducing idle time during fixed-period transaction layer packet processing. The link set of the operation parameters includes a counter for processing transaction layer packets for the fixed period, Dynamically determining the adaptive threshold according to the skip sequence set mode includes compensating for the predicted idle time based on the counter. The device according to claim 1.

10. The aforementioned device is a data storage device, A non-volatile storage medium configured to store data for the host system, The device according to claim 1, further comprising a peripheral interface connector configured to connect to a peripheral interface bus for storage communication with the host system using the peripheral interface protocol.

11. A computer implementation method, Using the peripheral interface protocol, receive data link packets from the host system via the peripheral interface, Determining errors in data link packets received via the aforementioned peripheral interface, Dynamically determining an adaptive threshold for selecting a replay mode based on a set of link operation parameters, Based on comparing the fill level of the replay buffer memory with the adaptive threshold, A selective replay mode for requesting retransmission of only the data link packets containing the aforementioned error, Selecting a replay mode from a sequential replay mode for requesting the retransmission of the erroneous data link packet and subsequent data link packets transmitted after the erroneous data link packet, Based on the selected replay mode, a replay request is sent to the host system. Methods that include...

12. Depending on the selective replay mode, The error-free sequential data link packets received after the error-ridden data link packet are stored in the replay buffer memory. In response to receiving an error-free version of the erroneous data link packet from the host system, the replay buffer memory sequentially transmits the error-free version of the erroneous data link packet and the transaction layer content from the error-free sequential data link packet. Clearing error-free sequential data link packets from the replay buffer memory, Depending on the sequential replay mode, Discard sequential data link packets received after the aforementioned erroneous data link packet, The computer implementation of claim 11, further comprising receiving the error-free version of the error-containing data link packet and a retransmitted sequential data link packet received after the error-free version of the error-containing data link packet, and sequentially transmitting the transaction layer content from the error-free version of the error-containing data link packet and the retransmitted sequential data link packet.

13. The method further includes periodically determining the adaptive threshold from a plurality of maximum fill levels based on the set of link operation parameters, The replay buffer memory has a buffer size corresponding to the total number of data link packets that can be stored in the replay buffer memory at one time. The fill level corresponds to the current number of data link packets stored in the replay buffer memory for pending selective replay requests. The adaptive threshold corresponds to a maximum fill level smaller than the buffer size of the replay buffer memory. The computer implementation method according to claim 11.

14. Determining that a flow control mode is active, the peripheral interface protocol supports the flow control mode, which is configured to change the memory size and format of the data link packets. The fill level of the replay buffer memory is determined based on the memory size and format of the data link packet in the flow control mode. Based on the flow control mode, the adaptive threshold is dynamically determined, The computer implementation method according to claim 13, further comprising:

15. In response to the detection of each erroneous data link packet, The computer implementation method according to claim 11, further comprising recalculating the adaptive threshold based on the current set of link operation parameters in order to dynamically determine the adaptive threshold for the erroneous data link packet.

16. Processing the set of link operation parameters via a statistical model that includes the set of link operation parameters and corresponding weight coefficients configured to balance the likelihood of successfully recovering the erroneous data link packet with the data link layer latency. The computer implementation method according to claim 11, further comprising:

17. The set of link operation parameters is: Current link speed and The number of lanes, The computer implementation method according to claim 11, comprising at least one link connection parameter selected from the current link speed and the bit error rate with respect to the number of lanes.

18. The set of link operation parameters is: Current data link packet workload volume and Host system historical workload volume and The computer implementation method according to claim 11, comprising a predicted workload amount and at least one link workload parameter selected from.

19. The aforementioned peripheral interface protocol supports a skip sequence set mode configured to periodically resynchronize the data link layer by introducing idle time during fixed-period transaction layer packet processing. The link set of the operation parameters includes a counter for processing transaction layer packets for the fixed period, Dynamically determining the adaptive threshold according to the skip sequence set mode includes compensating for the predicted idle time based on the counter. The computer implementation method according to claim 11.

20. Peripheral interface circuit, A data link layer configured to receive data link packets from a host system, To support selective replay mode, a replay buffer memory is configured to store received data link packets following error packets, Means for determining errors in data link packets received from the host system, Means for dynamically determining an adaptive threshold for selecting a replay mode based on a set of link operation parameters, Based on comparing the fill level of the replay buffer memory with the adaptive threshold, The selective replay mode for requesting retransmission of only the data link packets containing the aforementioned error, Means for selecting a replay mode from a sequential replay mode for requesting the retransmission of the erroneous data link packet and subsequent data link packets transmitted after the erroneous data link packet, Means for sending a replay request to the host system based on the selected replay mode, A peripheral interface circuit equipped with this.

Images