Accelerating link negotiation using control bus communications

Abstract

Systems and methods are described for sharing configuration data by devices connected by a control bus to accelerate physical link negotiation for a peripheral interface. A computer device, such as a data storage device, may include a peripheral interface configured to connect to a host system and a control bus interface connected to a control bus. Other devices on the same control bus may establish peer-to-peer communications through the control bus interface to share configuration data, such as physical link negotiated coefficients of the peripheral interface. To accelerate reestablishment of communications through the peripheral interface, the device may receive previously stored configuration data from another device through the control bus.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Non-Provisional Application No. 18 / 354,168, filed with the U.S. Patent and Trademark Office on July 18, 2023, entitled “Using Control Bus Communication to Accelerate Link Negotiation,” and is hereby incorporated by reference in its entirety for all purposes, which U.S. Non-Provisional Application claims priority to U.S. Provisional Application No. 63 / 380,973, filed on October 26, 2022. Technical Field

[0003] The present disclosure relates generally to computing system devices interconnected by a low-bandwidth bus, and more particularly to accelerating physical link negotiation of peripheral interfaces using communications over a low-bandwidth control bus. Background Art

[0004] Some computing systems, such as storage arrays, may include multiple storage devices that support one or more host systems through a peripheral or storage interface bus, such as Peripheral Component Interconnect Express (PCIe), Serial Advanced Technology Attachment (SATA), or Serial Attached [Small Computer Serial Interface (SCSI)] (SAS). These interfaces may be configured for high bandwidth transfer rates, such as 3 Gigabits (Gb) / second (s), 6 Gb / s, 8 Gb / s, 12 Gb / s, 16 Gb / s, etc., to enable fast and reliable data transfer to and from the storage devices, with some versions supporting up to 512 Gb / s.

[0005] Multi-device storage systems utilize multiple discrete storage devices, typically disk drives (solid-state drives, hard disk drives, hybrid drives, tape drives, etc.) for storing large amounts of data. These multi-device storage systems are typically arranged into arrays of drives interconnected by a common communication structure and, in many cases, controlled by a storage controller, a redundant array of independent disks (RAID) controller, or a general purpose controller for coordinating storage and system activities across the array of drives. The data stored in the array may be stored according to defined RAID levels, combinations of RAID modes, or other configurations for providing the desired data redundancy, performance, and capacity utilization. Generally speaking, these data storage configurations may involve some combination of redundant copies (mirroring), data stripping, and / or parity (calculated and stored), and may incorporate other data management, error correction, and data recovery processes that are sometimes specific to the type of disk drives used (e.g., solid-state drives versus hard disk drives).

[0006] Each storage device in a multi-device storage system may be connected to a host system through at least one high-bandwidth interface, such as PCIe, using an appropriate storage protocol for the storage device, such as solid-state drives (SSDs) or non-volatile memory express (NVMe) for accessing an all-flash array of storage blades. Some multi-device storage systems employ storage devices that are capable of communicating with each other and / or with a host system over an interconnect fabric and / or a network fabric through a high-bandwidth interface. Such fabric-based distributed storage systems may include storage devices configured with direct memory access to enable more efficient transfer of data to and from hosts and other systems.

[0007] In some configurations, each storage device may also include another interface configured for power management and / or low-bandwidth communication with a computing device sharing the same utility or control bus. For example, the storage device may include a control bus interface that complies with an internal integrated circuit (I2C), an improved internal integrated circuit (I3C), a system management bus (SMBus), or a similar low-bandwidth control bus protocol. These command buses can interconnect the storage devices in a multi-device storage system and provide a synchronous, multi-master, packet-switched serial communication bus. In some computing systems, control bus pins may be included as part of a physical peripheral device interface connector. Therefore, for example, a PCIe physical interface connector may include a set of I2C, I3C, or SMBus pins at one end of a connector for providing a low-bandwidth control interface. For comparison with a high-bandwidth peripheral device interface for data transmission, these low-bandwidth control bus interfaces can support approximately 3 to 30 megabits (Mb) / s and can operate at as low as 100 kilobits (kb) / s.

[0008] In some systems, a low-bandwidth control bus may be initialized for early communications in the boot cycle of a storage device to assist with power management, discovery, and access to external resources such as temperature sensors, fan controllers, light-emitting diode (LED) indicators, etc. The storage device may have access to the control bus before establishing host communications or control over a high-bandwidth peripheral bus.

[0009] In order to reestablish communications over a high-bandwidth peripheral bus, configuration parameters for physical link connections that support higher-level communications may need to be negotiated. Negotiating physical link connections each time a device cold boots, reboots, changes power modes, and / or experiences a link error takes time and slows performance of a multi-device storage system.

[0010] It may be advantageous to speed up the negotiation of physical link parameters for reestablishing communications over a high bandwidth peripheral bus.An efficient and reliable way to back up link configuration data to other connected devices and recall that configuration data to speed up link negotiation may be desired. Summary of the invention

[0011] Various aspects are described for using control bus communications to accelerate link negotiation, particularly messaging between connected devices over a low bandwidth control bus to share configuration data such as sets of gain and filter coefficient values ​​for peripheral interfaces of the connected devices.

[0012] A general aspect includes a system comprising: a first device, the first device comprising: a peripheral interface, the peripheral interface configured to connect to a host system using a peripheral interface protocol; a control bus interface, the control bus interface configured to connect to a control bus using a control bus protocol, wherein at least a second device is configured to connect to the control bus; a configuration service, the configuration service configured to receive configuration data for the peripheral interface protocol from the second device through the control bus interface; and a link state machine, the link state machine configured to use the configuration data received from the second device to reestablish communication to the host system through the peripheral interface. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each of which is configured to perform the actions of the method.

[0013] Specific implementations may include one or more of the following features. The first device may also include a control bus messaging service configured to: establish communication with a second device via a control bus; transmit a first peer message including configuration data to the second device during a first operating cycle; and transmit a second peer message to the second device during a second operating cycle to request the configuration data. The configuration service may be further configured to determine the configuration data based on establishing communication to the host system via the peripheral interface during the first operating cycle; the second operating cycle may be after the first operating cycle; and reestablishing communication via the peripheral interface may be during the second operating cycle. The second operating cycle may be in response to a connection loss event of the peripheral interface, the connection loss event being selected from: a power cycle event; a cold boot event; a reboot event; a reset event; a power state change event; and a link error event. The first device may also include a control bus messaging service configured to: establish communication with a plurality of connected devices including the second device; and broadcast a first peer message including configuration data to the plurality of connected devices, wherein each of the plurality of connected devices is configured to store the configuration data. The first device and each of the plurality of connected devices may be configured to store a connected device registry; each connected device registry may include a connected device entry corresponding to each of the plurality of connected devices and the first device; and each connected device entry may include: a connected device identifier; and corresponding configuration data. The configuration data may include a set of coefficient values ​​for physical link training, and the link state machine may be further configured to use the configuration data to negotiate physical link training between the first device and the host system to reestablish communication through the peripheral interface according to the peripheral interface protocol. The set of coefficient values ​​may include coefficient presets of physical link filter values ​​selected from a plurality of coefficient presets defined by the peripheral interface protocol. The system may include a plurality of data storage devices configured to: communicate with storage of the host system via a peripheral interface bus using the peripheral interface protocol; and communicate with peer data storage devices of each of the plurality of data storage devices via a control bus using the control bus protocol, wherein the first device is a first data storage device of the plurality of data storage devices, and the second device is a second data storage device of the plurality of data storage devices.The first device may be a data storage device, and the second device may be selected from: a host system, the host system including: a host control bus interface, the main host control bus interface configured for communication with the data storage device via a control bus using a control bus protocol; and a basic input output system, the main basic input output system configured to store configuration data from a previous operation cycle of the data storage device; and a baseboard management controller, the baseboard management controller including: a management control bus interface, the management control bus interface configured for communication with the data storage device via a control bus using a control bus protocol; and a management memory configured to store configuration data from a previous operation cycle of the data storage device. The system may include a plurality of data storage devices configured for: storage communication with the host system via a peripheral interface bus using a peripheral interface protocol; and control bus communication with the host system via a control bus using a control bus protocol, wherein during initialization of the data storage device, the control bus communication is established before the storage communication. The system may include a host system comprising: a host control bus interface configured to communicate with multiple data storage devices via a control bus using a control bus protocol; and a memory configured to store configuration data from a previous operation cycle of that data storage device for each of the multiple data storage devices, wherein: a first device is a data storage device from the multiple data storage devices, and a second device is the host system.

[0014] Another general aspect includes a computer-implemented method comprising: connecting, by a first device, to a control bus using a control bus protocol, wherein at least a second device is configured to connect to the control bus; receiving configuration data for a peripheral interface protocol from the second device via the control bus; initiating, by the first device, a connection to a host system via a peripheral interface using the peripheral interface protocol; and reestablishing communication to the host system via the peripheral interface using the configuration data received from the second device.

[0015] Specific implementations may include one or more of the following features. The computer-implemented method may include: determining configuration data based on establishing communication to a host system through a peripheral interface during a first operating cycle; establishing communication with a second device through a control bus by a first device; transmitting a first peer message including configuration data from the first device to the second device during the first operating cycle; and transmitting a second peer message to the second device during a second operating cycle to request configuration data, wherein the second operating cycle is after the first operating cycle; and reestablishing communication through the peripheral interface is during the second operating cycle. The computer-implemented method may include: determining, by the first device, a connection loss event of the peripheral interface, wherein the connection loss event may be selected from: a power cycle event; a cold boot event; a reboot event; a reset event; a power state change event; and a link error event; and the second operating cycle may be in response to the connection loss event. The computer-implemented method may include: establishing communication with a plurality of connected devices including the second device by a first device; broadcasting a first peer message including configuration data to the plurality of connected devices by the first device; and storing configuration data by each of the plurality of connected devices. The computer-implemented method may include: storing a connected device registry by a first device and each connected device in a plurality of connected devices, wherein: each connected device registry includes a connected device entry corresponding to each device in the plurality of connected devices and the first device; and each connected device entry includes: a connected device identifier; and corresponding configuration data. The computer-implemented method may include using configuration data to negotiate physical link training between the first device and the host system according to a peripheral interface protocol to reestablish communication through the peripheral interface, wherein the configuration data may include a set of coefficient values ​​for physical link training. The set of coefficient values ​​may include coefficient presets of physical link filter values ​​selected from a plurality of coefficient presets defined by the peripheral interface protocol. The first device may be a first data storage device in a plurality of data storage devices, the first data storage device being configured for: peer communication through a control bus; and storage communication with the host system through a peripheral interface bus; and the second device may be a second data storage device in a plurality of data storage devices. The computer-implemented method may include: receiving, by a host system, configuration data from a first device via a control bus during a first operating cycle; storing the configuration data in a memory of the host system during the first operating cycle; and transmitting, by the host system, the configuration data to the first device via the control bus during a second operating cycle before reestablishing communication via a peripheral interface.

[0016] Yet another general aspect includes a storage system comprising a plurality of data storage devices, and each data storage device comprising: a storage interface configured to connect to a host system; a control bus interface configured to connect to a control bus; a storage medium configured to store host data; a component for receiving configuration data for a storage interface protocol from another data storage device among the plurality of data storage devices through the control bus interface; a component for initiating a connection to the host system through the storage interface using the storage interface protocol; and a component for reestablishing communication to the host system through the storage interface using the configuration data received from another data storage device among the plurality of data storage devices.

[0017] Various embodiments advantageously apply the teachings of storage devices, multi-device storage systems, and / or other computer peripherals to improve the functionality of such computer systems. Various embodiments include operations that overcome or at least reduce problems previously encountered in storage arrays and / or systems, thereby being more reliable and / or efficient than other computing systems. That is, various embodiments disclosed herein include hardware and / or software with functionality to improve link connection times of peripheral interfaces (such as by sharing configuration data using messaging through a control bus between connected devices). Thus, the embodiments disclosed herein provide various improvements to storage networks and / or storage systems.

[0018] It should be understood that the language used in this disclosure has been selected primarily for readability and instructional purposes, and is not intended to limit the scope of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A multi-device storage system having a peripheral storage interface bus and a control bus is schematically illustrated.

[0020] Figure 2A A prior art state machine for peripheral interface link training is schematically illustrated.

[0021] Figure 2B Schematically illustrated as Figure 2A Recovery of high throughput communications of portions of a prior art state machine.

[0022] Figure 3 The phased negotiation of physical link coefficients of a peripheral interface using previous coefficient values ​​received from a connected device is schematically illustrated.

[0023] Figure 4 Schematically shows Figure 1 A host node in a multi-device storage system.

[0024] Figure 5More detailed schematic illustration Figure 1 Some components of a data storage device.

[0025] Figure 6 is a flow chart of an example method for accelerating link training using stored coefficient values.

[0026] Figure 7 is a flow chart of an example method for accelerating link training using stored configuration data.

[0027] Figure 8 is a flow chart of an example method for storing configuration data to a connected device via a control bus.

[0028] Fig. 9 is a flow chart of an example method for a connected device to receive and return configuration data via a control bus. DETAILED DESCRIPTION

[0029] Figure 1 An embodiment of an example data storage system 100 having a plurality of data storage devices 120 interconnected by both a peripheral interface bus 108 and a control bus 110 is shown. Although some exemplary features are shown, various other features are not shown for the sake of brevity and in order not to obscure the relevant aspects of the exemplary embodiments disclosed herein. To this end, as a non-limiting example, the data storage system 100 includes one or more data storage devices 120 (sometimes also referred to as information storage devices, storage devices, disk drives or drives). In some embodiments, the storage device 120 may be configured in a server or storage array blade or similar storage unit for use in a data center storage rack or chassis. The storage device 120 may be connected to one or more hosts 102 and provide data storage and retrieval capabilities for or through those host systems. In some embodiments, the storage device 120 may be configured in a storage hierarchy that includes storage nodes, storage controllers, and / or other intermediate components between the storage device 120 and the host 102. For example, each storage controller may be responsible for a corresponding set of storage nodes and their corresponding storage devices connected through a corresponding backplane network, although only the storage device 120 and the host 102 are shown.

[0030] In the illustrated embodiment, a plurality of storage devices 120 are attached to a common peripheral interface bus 108 for host communication. For example, the storage device 120 may include a plurality of drives arranged in a storage array, such as a storage device of a common rack, unit or blade in a shared data center or an SSD in an all-flash array. In some embodiments, the storage device 120 may share a backplane network, a network switch and / or other hardware and software components accessed by the peripheral interface bus 108 and / or the control bus 110. For example, the storage device 120 may be connected to the peripheral interface bus 108 and / or the control bus 110 via a plurality of physical port connections that define physical, transport and other logical channels for establishing communication with different components and subcomponents for establishing a communication channel to the host 102. In some embodiments, the peripheral interface bus 108 may be configured as a storage interface bus and provide a main host interface for storage device management and host data transfer, and the control interface bus 110 may include limited connectivity to the host for low-level control functions.

[0031] In some embodiments, the storage devices 120 may be referred to as a peer group or peer storage devices because they are interconnected via the storage interface bus 108 and / or the control bus 110. In some embodiments, the storage devices 120 may be configured for peer communications among the storage devices 120 via the peripheral interface bus 108 with or without assistance from the host 102. For example, the storage devices 120 may be configured for direct access to memory using one or more protocols such as Non-Volatile Memory Express (NVMe), Remote Direct Memory Access (RDMA), NVMe over Fabric (NVMeOF), etc. to provide command messaging and data transfer between storage devices using a high bandwidth storage interface and the storage interface bus 108.

[0032] In some embodiments, the storage device 120 may be configured for peer-to-peer communication using multi-master discovery and messaging that conforms to a low-bandwidth interface standard. For example, the storage device 120 may be configured for packet-based message delivery over the control bus 110 using a low-bandwidth bus protocol such as an inter-integrated circuit (I2C), an improved inter-integrated circuit (I3C), a system management bus (SMBus), etc. The storage device 120 may be interconnected via a common control bus to provide a low-bandwidth communication channel with the host 102 and other system components to assist in power management, discovery, and access to external resources such as temperature sensors, fan controllers, light-emitting diode (LED) indicators, etc. For example, the control bus 110 may connect the storage device 120 to a baseboard management controller (BMC) 106 to monitor the physical and power status of the storage device 120 of the host 102. In some embodiments, the BMC 106 may be embodied in a separate host or controller device to provide an integrated platform services interface and management of power and physical status across one or more storage systems. For example, the BMC 106 may include a dedicated management processor 106.1, a management memory 106.2, and a management control bus interface 110.2. The storage device 120 may be defined as a peer storage device based on its connection to the shared control bus 110. In some configurations, each device connected to the control bus 110 may be referred to as a connected device, which may include the storage device 120, the host 102, and / or the BMC 106. Figure 1 Other devices (not shown) may also be connected devices in the data storage system 100, such as fan controllers, sensor arrays, network interface controllers, processor devices (e.g., graphics processor units), and / or other devices managed by the BMC 106.

[0033] In some embodiments, the data storage device 120 is or includes a solid state drive (SSD). Each data storage device 120.1-120.n may include a non-volatile memory (NVM) or device controller 130 based on computing resources (processor and memory) and multiple NVM or media devices 140 for data storage (e.g., one or more NVM devices, such as one or more flash memory devices). In some embodiments, a corresponding data storage device 120 in one or more data storage devices includes one or more NVM controllers, such as a flash memory controller or a channel controller (e.g., for a storage device with NVM devices in multiple memory channels). In some embodiments, the data storage devices 120 may each be encapsulated in a housing, such as a multi-part sealed housing with a defined form factor and ports and / or connectors for interconnecting with the peripheral interface bus 108 and / or the control bus 110.

[0034] In some embodiments, the corresponding data storage device 120 may include a single media device, while in other embodiments, the corresponding data storage device 120 includes multiple media devices. In some embodiments, the media device includes a NAND-type flash memory or a NOR-type flash memory. In some embodiments, the data storage device 120 may include one or more hard disk drives (HDDs). In some embodiments, the data storage device 120 may include a flash memory device, which in turn includes one or more flash memory dies, one or more flash memory packages, one or more flash memory channels, etc. However, in some embodiments, one or more data storage devices in the data storage device 120 may have other types of non-volatile data storage media (e.g., phase change random access memory (PCRAM), resistive random access memory (ReRAM), spin transfer torque random access memory (STT-RAM), magnetoresistive random access memory (MRAM), etc.).

[0035] In some embodiments, each storage device 120 includes a device controller 130, which includes one or more processing units (sometimes also referred to as CPUs or processors or microprocessors or microcontrollers) configured to execute instructions in one or more programs. In some embodiments, the one or more processors are shared by one or more components within the functionality of the device controller and in some cases beyond its functionality. The media device 140 is coupled to the device controller 130 via a connector that typically transmits commands in addition to data, and optionally transmits metadata, error correction information, and / or other information in addition to transmitting data values ​​to be stored in the media device and data values ​​read from the media device 140. The media device 140 may include any number (i.e., one or more) of memory devices, including but not limited to non-volatile semiconductor memory devices, such as flash memory devices.

[0036] In some embodiments, the media device 140 in the storage device 120 is divided into a plurality of addressable and individually selectable blocks, sometimes referred to as erase blocks. In some embodiments, the individually selectable blocks are the smallest erasable units in the flash memory device. In other words, each block contains the minimum number of memory cells that can be erased simultaneously (i.e., in a single erase operation). Each block is typically further divided into a plurality of pages and / or word lines, wherein each page or word line is typically an instance of the smallest individually accessible (readable) portion in the block. However, in some embodiments (e.g., using certain types of flash memory), the smallest individually accessible unit of a data set is a sector or codeword, which is a subunit of a page. That is, a block includes a plurality of pages, each page includes a plurality of sectors or codewords, and each sector or codeword is the smallest data unit for reading data from a flash memory device.

[0037] A data unit may describe any size allocation of data, such as a host block, data object, sector, page, multi-slice page, erase / program block, media device / package, etc. A storage location may include a physical and / or logical location on the storage device 120 and may be described and / or allocated at different levels of granularity depending on the storage medium, storage device / system configuration, and / or context. For example, for host read and write purposes, a storage location may be allocated by host logical block address (LBA) data unit size and addressability, but in other contexts managed as a page with storage device addressing managed in the media flash translation layer (FTL). A media segment may include a physical storage location on the storage device 120, which may also correspond to one or more logical storage locations. In some embodiments, a media segment may include a series of consecutive physical storage locations, such as adjacent data units on a storage medium, and for a flash memory device, the media segment may correspond to one or more media erase or program blocks. A logical data group may include multiple logical data units that may be grouped on a logical basis regardless of the storage location, such as a data object, a file, or other logical data structure consisting of multiple host blocks.

[0038] In some embodiments, a host or host system 102 may be coupled to the data storage system 100 via a network interface as part of a host fabric network including a storage interface bus 108 as a host fabric interface. Figure 1The fabric network may include a plurality of host systems 102 (only one of which is shown) coupled to the data storage system 100 via a fabric network, which may be a storage network interface or other interface capable of supporting communication with multiple host systems 102. The fabric network may include wired and / or wireless networks (e.g., any number and / or configuration of public and / or private computer networks) that may be coupled in a suitable manner for transmitting data. For example, the fabric network may include any means of a conventional data communication network, such as a local area network (LAN), a wide area network (WAN), a telephone network such as a public switched telephone network (PSTN), an intranet, the Internet, or any other suitable communication network or combination of communication networks.

[0039] The host system 102 or the corresponding host in the system with multiple hosts can be any suitable computer device, such as a computer, a computer server, a laptop computer, a tablet device, a netbook, an Internet kiosk, a personal digital assistant, a mobile phone, a smart phone, a gaming device, or any other computing device. The host system 102 is sometimes referred to as a host, a client, or a client system. In some embodiments, the host system 102 is a server system, such as a server system in a data center, or a storage system, such as a storage array in a data center. In some embodiments, one or more host systems 102 can be one or more host devices different from a storage controller or a storage node accommodating multiple storage devices 120. One or more host systems 102 can be configured to store and access data in multiple storage devices 120.

[0040] The host system 102 may include one or more central processing units (CPUs) 104 for performing computing operations or instructions for accessing the storage device 120 through the peripheral interface bus 108. In some embodiments, the CPU 104 may include a host processor and is associated with an operating memory 112 for performing both storage operations and storage interface protocols compatible with the peripheral interface bus 108 and the storage device 120. In some embodiments, the memory 112 may include one or more dynamic random access memory (DRAM) devices for the storage device 120 to use for storing and transferring commands, management parameters and / or host data. In some embodiments, the storage device 120 may be configured for direct memory access (DMA) such as using a remote direct memory access (RDMA) protocol through the peripheral interface bus 108 to access and use the host memory buffer assigned to that storage device. The memory 112 may also include an operating system (not shown), firmware and / or software applications for performing various functions of the host system 102. In some configurations, the memory 112 may include a host basic input-output system (BIOS) 112.1 for establishing basic communications between components of the host system 102 during boot up. In some embodiments, the host BIOS 112.1 may have access to the control bus 110 through the host control bus interface 110.1 and may be configured to pass messages between connected devices before establishing communications through the peripheral interface bus 108.

[0041] In some embodiments, a separate storage interface unit or controller (not shown) may provide storage interface protocols and associated processor and memory resources. The host system 102 may include interface ports for connecting to the peripheral interface bus 108 and / or the control bus 110. For example, the host system 102 may include a peripheral bus interface 108.1 and a control bus interface 110.1 for physically connecting to the corresponding peripheral interface bus 108 and control bus 110. The host 102, the peripheral interface bus 108, and / or the storage interface unit may include a root complex 118 for the peripheral interface protocol, which manages connections and communications through the peripheral interface bus 108. For example, the root complex 118 may be configured as a PCIe root complex that connects the CPU and memory subsystem to a PCIe switch fabric composed of various endpoints, which may include storage devices 120, the host system 102, and other devices connected to the peripheral interface bus 108. In some embodiments, the host system 102 may provide processor and memory resources to support the root complex 118. From the perspective of the storage device 120 , the peripheral interface bus 108 may be referred to as a host interface bus and provides a host data path between the storage device 120 and the host 102 .

[0042] The host system 102 may include a BMC 106 configured to monitor the physical state of the host 102, storage device 120, and / or other components of the data storage system 100. In some embodiments, the BMC 106 may include a management processor 106.1, a management memory 106.2, a management control bus interface 110.2, sensors, fan controllers, and other resources integrated into the BMC 106 and / or accessible via the control bus 110. The BMC 106 may be configured to measure internal variables such as temperature, humidity, power supply voltage, fan speed, communication parameters, and / or operating system (OS) functions within the housing, adjacent components, and / or from the components themselves within the host 102 or data storage system 100. The BMC 106 may enable the system and components to be power cycled or rebooted as needed via control signals on the control bus 110. In some embodiments, the BMC 106 may be configured to receive status communications from the storage device 120 via the control bus 110 during the boot cycle before initiating host communications via the peripheral interface bus 108.

[0043] In some embodiments, the data storage system 100 includes one or more processors, one or more types of memory, a display and / or other user interface components such as a keyboard, a touch screen display, a mouse, a trackpad, and / or any number of supplementary devices that add functionality. In some embodiments, the data storage system 100 does not have a display and other user interface components.

[0044] Figure 2A A schematic representation of an example state machine 200 for establishing link communication across a peripheral interface bus using a peripheral interface protocol such as PCIe is shown. For example, the state machine 200 may include a link training and status state machine (LTSSM) configured to enable two devices to exchange training data to negotiate multiple link parameters, such as channel polarity, number of channels, equalization, data rate, etc., so as to establish a physical data link that meets a target bit error rate to support high throughput data signals and bidirectional communication through a physical peripheral interface connection such as a peripheral interface bus. Link training establishes analog signal transmission and corresponding logical encoding across the peripheral interface bus for sending data bits in the analog signal. Once the physical link training is established and the expected bit error rate is met, higher-level communications can be achieved through a data link layer for managing data packets, flow control, and confirmation. The data link layer supports a transaction layer for various configurations, message passing, and memory or input / output (I / O) transactions. The data link layer supports an application layer for encapsulating data in a payload of a transaction. For example, at the application layer, a storage interface protocol specific to a data storage device, such as an NVMe command, can be implemented.

[0045] In the detection state 210, the state machine 200 detects when a device is present on a physical connection. In the polling state 212, the state machine 200 sends a training ordered set of data and responds to receiving a trained ordered set. In this state, a bit lock and a symbol lock are established, just as in the configuration of channel polarity. In the configuration state 214, the state machine 200 enables two end point devices, i.e., a transmitter and a receiver, to transmit and receive data at a target data rate. During this state, the port can be configured as a link with a configured and negotiated channel and width. In the link 0 (L0) state 216, the normal operating state of the physical link layer is established by the state machine 200. During this state, data and control packets from higher layers can be sent and received, and application-level communication between devices can be achieved. In the link 0 power saving state (L0s) 218, the physical link can enter and recover from a power saving state. In some configurations, additional power saving states may be possible and accessible through the L0s state 218. In the recovery state 220, the two devices, i.e., the transmitter and the receiver, may be transmitting and receiving data at the previously supported data rate using the configured link and channel configurations, but may be experiencing a bit error rate that does not support reliable communication at the supported data rate. In this state, the devices may change the data rate, reestablish bit lock and symbol lock, change block alignment, and adjust channel-to-channel de-skew. Additional states and state transitions (not shown) may be supported by the state machine 200.

[0046] like Figure 2BAs shown, the recovery state 220 may include a recovery sub-state machine that manages the recovery state and exits back to various states in the state machine 200. At the recovery entry 230, the recovery sub-state machine is initiated to attempt to change the configuration of the communication channel. The recovery lock sub-state 232 enables the device to exchange training data sequences to rebuild and test bit locks and symbol locks. The recovery configuration state 234 enables the device to rebuild and test link and channel configurations. The recovery idle sub-state 236 enforces a timeout period before exiting the recovery state at the recovery exit 238. Depending on the entry and exit conditions of the recovery state 220, the recovery exit 238 can return to any state in the state of the state machine 200. The recovery equalization sub-state 240 enables the device to train the electrical parameters of the link for a specific endpoint pair. For example, the recovery equalization 240 can determine the set of coefficient values ​​negotiated between the two devices for setting the physical link filter values ​​of their corresponding equalization filters on their transmitters and receivers. In some embodiments, the peripheral interface protocol can define a series of preset values ​​of transmitter and receiver equalizer coefficients, including a set of default filter values ​​used to initialize the training process. For example, the peripheral interface protocol may define a default set of coefficient values ​​and a series of additional sets of preset coefficient values, and the training process may increment by the presets until the selected coefficient values ​​achieve the desired bit error rate at the selected data rate. The link training process used in recovery equalization 240 may be described below with respect to Figure 3 Further described. Recovery speed 242 may include a data rate speed test across the link to determine if the desired data rate is being achieved (with an acceptable error rate).

[0047] The recovery state 220 may enable the state machine 200 to manage speed changes, link width changes, errors in the L0 state 216, returns from a low power link state (e.g., the L0s state 218), and / or recovery from the L0 state of a link that was not properly trained in an earlier iteration. In some embodiments, the recovery state 220 may be used to train the link connection through higher generations and corresponding throughputs to achieve the desired throughput of the peripheral interface connection. For example, each physical link may initially be trained to a throughput of 2.5 Gigatransfers / second (GT / s), and then the recovery state 220 may be used to change the throughput to 5 GT / s and return to the L0 state 216. The recovery state 220 may be used again to increment the throughput to 8 GT / s, and so on, to achieve a throughput supported by the configuration of the device for the generation of the peripheral interface protocol.

[0048] Figure 3 An example storage system 300 is shown that uses previous coefficient values ​​received from a connected device (such as Figure 1Schematic representation of the phased negotiation of physical link coefficients of a peripheral interface in a multi-device data storage system 100 in FIG. 1 . As described above, the storage device 120.1 and the host system 102 may be connected by a peripheral interface bus and periodically negotiate coefficient values ​​to establish physical link communication and support data links, transactions, and application-level communications across the peripheral interface. For example, the restore equalization substate 240 may initiate link training between the storage device 120.1 and the host system 102 to establish equalization parameters that obtain acceptable signal integrity that meets the desired bit error rate (BER) level. During link training, each side negotiates a set of coefficient values ​​for their respective transmitter and receiver equalizer circuits until the bit error rate meets a target threshold, such as 10-12 BER. In some configurations, link training may take a significant amount of time and is completed independently for each channel port of the peripheral interface. For example, finding the optimal coefficient value may take up to 24 milliseconds (mS) in each direction of each channel (and the peripheral bus configuration may support 16 or more channels). The storage device 120.1 and / or the host system 102 may be configured for control bus communications with one or more other connected devices and may be able to use the control bus communications to retrieve previously stored coefficient values ​​to reduce link training time.

[0049] In the configuration shown, storage device 120.1 is configured with a control bus connection and has stored coefficient values ​​from a previous operation cycle when the physical link is trained to an expected bit error rate for a target data throughput. The coefficient values ​​may have been stored to one or more other storage devices, such as storage devices 120.2 to 120.n, host BIOS 112.1, BMC 106, or another connected device with control bus messaging and local memory. Prior to link training, storage device 120.1 may perform a recall process 302 to retrieve previously stored coefficients. The recall process 302 may be triggered when storage device 120.1 boots, reboots, or recovers from a low power mode (where previously used coefficients are lost and, in some embodiments, volatile memory is cleared). At block 304, a set of coefficients may be requested from one or more connected devices to which the coefficients were previously transmitted for storage. For example, storage device 120.1 may have designated another storage device to back up its coefficients and may transmit a message over the control bus requesting a set of previously stored coefficient values ​​for a peripheral interface connection to host 102. At block 306, a set of previously stored coefficient values ​​may be received from a connected device via a control bus. For example, another storage device may respond with a message that includes the set of previously stored coefficient values ​​in the payload of the message. At block 308, link training may be initialized. For example, the coefficient recall process 302 may be completed before or in parallel with the link training state machine initializing the peripheral interface and going through various states to establish communications. When booting or waking up, control bus communications may be established before communications through the peripheral interface. The goal may be to receive the coefficients at block 306 before the link training state machine reaches physical link negotiation so that the recalled coefficients are available in the negotiation process 310. Note that entering physical link negotiation at blocks 320 and 322 may not be dependent on completing the recall process 302, and the negotiation process 310 may proceed without the recalled coefficients according to existing methods of physical link negotiation from default values.

[0050] The physical link negotiation process 310 may be configured in stages and performed between an upstream device (such as a root complex controlled by the host 102) and a downstream device (such as the storage device 120.1). At blocks 320 and 322, both devices will enter into a negotiation to determine a set of equalizer coefficients that achieve an acceptable bit error rate for the current data rate. For example, a link training state machine such as the state machine 200 may enter a recovery state and initiate a recovery equalization substate. In stage 312, such as stage 0 in some peripheral interface protocols, at block 324, the storage device 120.1 may verify the channel by checking that its receiver bit error rate is below a first BER threshold A (such as 10-4 BER). For example, the host system 102 may send a known training data set, and the storage device 120.1 may receive, decode and compare the received data to a known data sequence to determine a bit error rate and compare the bit error rate to a first BER threshold. At stage 314, such as in stage 1 in some peripheral interface protocols, at box 326, the host system 102 may verify the channel by checking that its receiver bit error rate is below a first BER threshold A (such as 10-4 BER). For example, the storage device 120.1 may send a known training data set, and the host system 102 may receive, decode and compare the received data with a known data sequence to determine a bit error rate and compare the bit error rate with a first BER threshold. At the end of stage 315, both devices have confirmed that the channel error rate is less than the BER threshold A and that a minimum level of data delivery and integrity has been met.

[0051] In stage 316, such as in stage 2 in some peripheral interface protocols, the downstream device may iteratively tell the upstream device to use a specific set of presets corresponding to the expected coefficient values ​​of the upstream device transmitter and then verify the channel until a second and lower BER threshold B, such as 10-12 BER, is met. By using the recalled set of coefficient values ​​from the recall process 302, the storage device 120.1 can speed up the negotiation of coefficient values, ideally to a single exchange, rather than iterating through the available presets. In some embodiments, coefficient values ​​may be determined for equalizers on both the transmitter side and the receiver side of the channel (note that both devices have both transmit and receive functionality). For example, the transmit side coefficients may include filter tap weights for determining de-emphasis and pre-impact of the transmitted analog signal, and the receive side coefficients may include parameters for continuous-time linear equalization and decision feedback equalization filters.

[0052] At block 330, coefficients may be selected for iteration of stage 316. For example, storage device 120.1 may select an initial set of coefficients to attempt to achieve BER threshold B. If recall process 302 has been completed, storage device 120.1 may select the set of coefficients received at block 306. If recall process 302 has not been completed, storage device 120.1 may select a default set of coefficients determined by the storage interface protocol. In some embodiments, the recalled set of coefficients may include receiver coefficients, and storage device 120.1 may use the recalled set of coefficients to set its receiver coefficients at block 330. At block 332, coefficients may be transmitted to host system 102 using a peripheral connection and compensating for a current channel error rate. For example, storage device 120.1 may transmit the recalled set of coefficients or a default set of coefficients (as actual values ​​or presets defined by reference) for use with a host-side transmitter. At block 334, host system 102 updates transmitter coefficients based on the set of coefficients. For example, the host system 102 may change the filter coefficients of the transmitter and send the training data set from the transmitter back to the storage device 120.1. At block 336, the bit error rate of the channel may be checked to see if the bit error rate is less than a BER threshold B. For example, the storage device 120.1 may receive, decode and compare the training data set to a known sequence of training data sets to determine the bit error rate, and then compare the bit error rate to the BER threshold B. If the BER threshold B is met, then the operation may proceed to stage 318. If the BER threshold B is not met, then the storage device 120.1 may iterate through blocks 330 to 336, selecting different sets of coefficients from an available range of values ​​until the BER threshold B is met. Ideally, the recalled coefficient values ​​may reduce the number of iterations to a single pass. However, if the previously stored coefficient values ​​are not available or they do not cause the BER threshold B to be met (such as due to a change in the physical configuration of the interconnect or other changes affecting the signal characteristics of the peripheral channel), then the normal iterative process may be used to negotiate the coefficients of the transmitter of the host system 102.

[0053] In stage 318, such as in stage 3 in some peripheral interface specifications, the upstream device may perform a similar process to train the coefficient values ​​of its receiver and the transmitter of the downstream device. Similarly, it may use a BER threshold B to evaluate whether the training is successful. In the illustrated embodiment, the host system 102 does not use the recalled coefficient values, but uses a default iterative training method. Note that in other embodiments, the host system 102 may include a control bus connection and may also benefit from recalling previously stored configuration data from other connected devices to accelerate this stage. At box 338, the coefficients may be transmitted to the storage device 120.1 using the peripheral connection and compensating for the current channel error rate. For example, the host 102 may transmit a default set of coefficients (as actual values ​​or presets defined by reference) or the next set of coefficients to attempt to use for the device-side transmitter. At box 340, the storage device 120.1 updates the transmitter coefficients based on the received set of coefficients from the host system 102. For example, the storage device 120.1 may change the filter coefficients of the transmitter and send the training data set back from the transmitter to the host system 102. At block 342, the bit error rate of the channel may be checked to see if the bit error rate is less than a BER threshold B. For example, the host system 102 may receive, decode, and compare a training data set to a sequence of known training data sets to determine a bit error rate, and then compare the bit error rate to the BER threshold B. If the BER threshold B is met, the channel error may be verified as being less than the BER threshold B at block 344, and the negotiation process 310 may end successfully. If the BER threshold B is not met, the host system 102 may iterate through blocks 338 to 342, selecting different sets of coefficients from the available range of values ​​until the BER threshold B is met.

[0054] Figure 4 A schematic representation of an example host system 102 is shown. The host system 102 may include a bus 410, a processor 420, a local memory 430, one or more optional input units 440, one or more optional output units 450, and a communication interface 460. The bus 410 may include one or more conductors that permit communication between components of the host 102. The processor 420 may include any type of conventional processor or microprocessor that interprets and executes instructions. The local memory 430 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by the processor 420 and / or a read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by the processor 420 and / or any suitable storage element such as a hard disk or a solid-state storage element. For example, Figure 1The host driver 116 in the local memory 430 may be instantiated in instructions, operations, or firmware stored in the local memory 430 for execution by the processor 420. The optional input unit 440 may include one or more conventional mechanisms that allow an operator to input information to the host 102, such as a keyboard, a mouse, a pen, a voice recognition and / or a biometric recognition mechanism, etc. The optional output unit 450 may include one or more conventional mechanisms that output information to the operator, such as a display, a printer, a speaker, etc. The communication interface 460 may include any transceiver-like mechanism that enables the host 102 to communicate with other devices and / or systems. In some embodiments, the communication interface 460 may include one or more peripheral device interfaces, such as a PCIe interface for connecting to the storage device 120.

[0055] Figure 5 Schematically illustrates selected modules of a storage device 500 configured to accelerate physical link training by backing up and recalling previously trained configuration data from a peer storage device such as storage device 120. In some embodiments, storage device 500 may use another connected device (such as a BIOS, or BMC, of ​​a host system) to back up and recall configuration data via a shared control bus. Storage device 500 may be combined with a storage device similar to Figures 1 to 3 For example, storage device 500 may be configured as a centralized storage device 120 of peer storage devices interconnected by a control bus and include a peripheral bus interface for communicating host data in a multi-device data storage system.

[0056] The storage device 500 may include a bus 510 interconnecting at least one processor 512, at least one memory 514, and at least one interface, such as a storage bus interface 516 and a control bus interface 518. The bus 510 may include one or more conductors that allow communication between the components of the storage device 500. The processor 512 may include any type of processor or microprocessor that interprets and executes instructions or operations. The memory 514 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by the processor 512 and / or a read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by the processor 512 and / or any suitable storage element such as a hard disk or a solid-state storage element.

[0057] The storage bus interface 516 may include a physical interface for connecting to a host using a peripheral interface protocol that supports storage device access. For example, the storage bus interface 516 may include a PCIe, SATA, SAS, or similar storage interface connector that supports NVMe access to solid-state media including non-volatile memory devices 520. The control bus interface 518 may include a physical interface for connecting to a control bus using a low-bandwidth interface protocol for low-level control messages between computing components. For example, the control bus interface 518 may include an I2C, I3C, SMBus, or a similar bus interface connector that supports component-to-component messaging (such as multi-master packet-based messaging over a two-wire bus).

[0058] The storage device 500 may include one or more non-volatile memory devices 520 configured to store host data. For example, the non-volatile memory device 520 may include a plurality of flash memory packages organized as an addressable memory array. In some embodiments, the non-volatile memory device 520 may include a NAND or NOR flash memory device composed of a single-level cell (SLC), a multi-level cell (MLC), or a triple-level cell.

[0059] The storage device 500 may include a plurality of modules or subsystems that are stored and / or instantiated in the memory 514 for execution by the processor 512 according to instructions or operations. For example, the memory 514 may include a host interface 530 configured to receive, process, and respond to host data requests from a client or host system. The memory 514 may include a non-volatile memory (NVM) controller 534 configured to manage read operations and write operations to the non-volatile memory device 520. The memory 514 may include a control bus protocol 540 configured to manage communications through the control bus interface 518 according to a determined protocol or standard. The memory 514 may include a configuration service 542 configured to support configuration of the storage interface protocol 532 and more specifically the underlying peripheral interface protocol for establishing application-level communications for the storage interface protocol by backing up and recalling configuration data across operating cycles. The memory 514 may include a peer sharing service 560 configured for communication between storage devices attached to the same control bus as the storage device 500 and for sharing configuration data. In some embodiments, a peer may be defined as any connected device that shares the same control bus. The memory 514 may include a peer backup service 570 configured to receive and store configuration data from peer devices to support their backup and recall of configuration data.

[0060] The host interface 530 may include an interface protocol and / or a set of functions and parameters for receiving, parsing, responding to, and otherwise managing host data requests from the host. For example, the host interface 530 may include functions for receiving and processing host requests for reading, writing, modifying, or otherwise manipulating data blocks and their corresponding client or host data and / or metadata according to host communication and storage protocols. In some embodiments, for host data units 520.3 stored in non-volatile memory devices 520, the host interface 530 may perform direct memory access and / or access using the NVMe protocol through the storage bus interface 516. For example, the host interface 530 may include a host communication protocol compatible with PCIe, SATA, SAS, and / or another bus interface that supports the use of NVMe and / or RDMA protocols for data access to host data 520.3.

[0061] In some embodiments, the host interface 530 may include a plurality of hardware modules and / or software modules configured to handle or manage the defined operations of the host interface 530 using the processor 512 and the memory 514. For example, the host interface 530 may include a storage interface protocol 532 configured to conform to the physical, transport and storage application protocols supported by the host for communicating through the storage bus interface 516. For example, the storage interface protocol 532 may include communications, commands, syntax functions, programs and data structures compatible with both PCIe and NVMe. In some embodiments, the storage interface protocol 532 may include an interface protocol for managing physical links or channels, data links and transaction layers to support application-level interactions in the payload of the interface protocol. For example, a peripheral interface protocol such as PCIe may define a communication channel between a storage device 500 and a host system through a storage bus interface 516 and a corresponding peripheral interface bus. In some embodiments, the host interface 530 may include additional modules (not shown) for command handling, buffer management, storage device management and reporting and other host-side functions.

[0062] In some embodiments, the storage interface protocol 532 may include a link state machine 532.1 that is configured to manage the establishment and negotiation of physical links and data links through the storage bus interface 516. For example, the link state machine 532.1 may be substantially as described with respect to Figure 2A The state machine 200 in Figure 2B The recovery status 220 and Figure 3532.1. The link state machine 532.1 may include physical link negotiation to establish a physical and data link that supports a desired data rate that satisfies (equal to or below) a predefined BER threshold for the storage bus interface 516. In some embodiments, physical link training may be based on determining configuration data for a receiver and a transmitter that transmit analog data signals across the peripheral interface bus. For example, each link, path, and / or channel may include a receiver filter and a transmitter filter (in both directions) having coefficients configured to establish data signal integrity that satisfies a predefined BER threshold. These sets of coefficient values ​​may be determined and verified by the link state machine 532.1 whenever the physical and data links are established, such as at power-up, wake-up, or error recovery. In some embodiments, the link state machine 532.1 may include or interface with coefficient value logic 532.2, which is configured to determine a set of coefficient values ​​for initializing the negotiation and each iteration through the negotiation process (if necessary). For example, the link state machine 532.1 may include default logic for selecting and iterating through a set of coefficient values ​​during the negotiation process. In some embodiments, coefficient value logic 532.2 may be configured to check recalled configuration data stored in memory 514 by configuration service 542. For example, coefficient value logic 532.2 may check registers managed by configuration service 542 for previously used coefficient values. In some embodiments, coefficient value logic 532.2 may use a sequence of coefficient presets 532.3 to determine an initial set of coefficient values ​​and iterate through other sets of coefficient values. For example, coefficient presets 532.2 may include a series of 11 or more preset sets of coefficient values ​​that may be incrementally tried during the negotiation process in the event that previously used coefficient values ​​are not available and / or do not successfully meet the BER threshold.

[0063] The NVM controller 534 may include an interface protocol and / or a set of functions and parameters for reading, writing, and deleting data units in the non-volatile memory device 520. For example, the NVM controller 534 may include functions for performing host data operations related to host storage commands received through the host interface 530. For example, a PUT or write command may be configured to write a host data unit to the non-volatile memory device 520. A GET or read command may be configured to read data from the non-volatile memory device 520. A DELETE command may be configured to delete data from the non-volatile memory device 520, or at least mark the location of the data to be deleted until a future garbage collection or similar operation actually deletes the data or reallocates the physical storage location for another purpose. In some embodiments, the NVM controller 534 may include flash translation layer (FTL) management, data state machine, read / write buffer management, NVM device interface protocol, NVM device configuration / management / maintenance, and other device-side functions. In some embodiments, the NVM controller 534 may be configured to allocate a portion of the memory locations in the non-volatile memory device 520 for storing data other than the host data 520.3. For example, the NVM controller 534 may allocate device data 520.1 as a memory location reserved for internal device data (including device configuration, parameters, and internal operating data). In some embodiments, the NVM controller 534 may allocate peer data 520.2 as a memory location reserved for internal device data received from and about peer storage devices (including internal operating data and / or configuration data from those peer storage devices). In some embodiments, the storage space allocated to device data 520.1 and / or peer data 520.2 may be excluded from the storage capacity available for host data 520.3, such as over-provisioned storage locations hidden from the host for storing internal operating data, FTL tables, replacing bad blocks, etc.

[0064] The control bus protocol 540 may include interfaces, functions and parameters for communication within a host or associated baseboard management controller (BMC) using multi-master discovery and messaging compatible with low-bandwidth interface standards. For example, the storage device 500 may be configured for packet-based messaging through the control bus interface 518 using a low-bandwidth bus protocol (such as I2C, I3C, SMBus, etc.). The control bus protocol 540 may include multiple hardware modules and / or software modules configured to use a processor 512, a memory 514 and a control bus interface 518 for communication with a host component (including a peer storage device on the same control bus) using a control bus protocol supported by the connected control bus. In some embodiments, the control bus protocol 540 may provide a low-bandwidth communication channel with the host and other system components to assist in power management, discovery and access to external resources, such as temperature sensors, fan controllers, light-emitting diode (LED) indicators, etc. For example, the control bus interface 518 may connect the storage device 500 to the BMC to monitor the physical state of the storage device 500. In some embodiments, the control bus protocol 540 may provide discovery and messaging functionality used by the peer sharing service 560 .

[0065] Configuration services 542 may include an interface protocol and a set of functions and parameters for supporting connection and establishment of host communications through storage bus interface 516 of storage device 500 using previously stored configuration data. For example, configuration services 542 may back up configuration data to other connected devices through control bus interface 518 so that the configuration data can be quickly recalled during subsequent boot, power mode or error recovery changes to accelerate physical link training and negotiation. More specifically, configuration services 542 may support link state machine 532.1 and coefficient value logic 532.2 with configuration data, such as a set of coefficient values ​​for link negotiation, when previous configuration data could not have been accessed by link state machine 532.1. Configuration services 542 may include multiple hardware and / or software modules configured to support storage interface protocol 532 using processor 512, memory 514 and peer sharing service 560. For example, configuration services 542 may include a configuration backup module 544 for determining and transmitting configuration data to be backed up. Configuration service 542 may include a configuration recall module 546 for recalling configuration data for use by storage interface protocol 532 .

[0066] The configuration backup module 544 may include data structures, functions, parameters, and / or interfaces for accessing or receiving configuration data from the storage interface protocol 532 and transmitting the configuration data to other connected devices through the control bus interface 518. For example, the configuration backup module 544 may access the current set of coefficient values ​​for the peripheral interface channel after successful link training and normal application-level communication have been established through the storage bus interface 516, and transmit them to the peer storage device for storage until needed. In some embodiments, the configuration backup module 544 may include a configuration data selector 544.1 that is configured to select configuration data to be backed up. For example, the configuration data selector 544.1 may include interfaces and functions for accessing configuration parameters stored by the storage interface protocol 532 (such as signal conditioning and data encoding parameters, including coefficient values ​​negotiated between the storage device 500 and the host system). In some embodiments, the backup condition 544.2 may include events, trigger states, and / or other operating conditions that determine when the configuration data selector 544.1 determines the set of configuration data to be backed up. For example, backup condition 544.2 may monitor link state machine 532.1 to enter an operational link state and maintain the link state for a predefined period of time during an operational cycle (to ensure that a stable link has been configured), and in response to determining a stable operational link, initiate configuration data selector 544.1 to determine a set of configuration data to be backed up. In some embodiments, backup device selector 544.2 may determine one or more connected devices to receive and store selected configuration data. For example, backup device selector 544.2 may be configured to select one or more peer storage devices and / or all peer storage devices to receive backup configuration data. In some embodiments, one or more peer storage devices, host systems (where the host system is a connected device to the control bus), BMCs, or other connected devices may be configured as dedicated backup devices and / or parts of a backup group, and a set of connected devices for use by configuration backup module 544 may be configured in backup device selector 544.3. In some embodiments, the configuration backup module 544 may include a backup message interface 544.4 that communicates the selected configuration data and the set of connected device identifiers of the backup device to the messaging service 566. For example, the backup message interface 544.4 may include an interface service for initiating the messaging service 566, wherein the service call from the backup message interface 544.4 acts as a sharing trigger event 566.2, the selected backup device completes the sharing peer selector parameters, and the selected configuration data completes the sharing content selector parameters.

[0067] The configuration recall module 546 may include data structures, functions, parameters and / or interfaces for requesting, receiving and managing configuration data of the storage interface protocol 532, which is recalled from other connected devices via the control bus interface 518. For example, the configuration recall module 544 may determine that previously stored configuration data is needed, trigger a recall message to be transmitted via the control bus, receive the recalled data and provide it to the storage interface protocol 532. In some embodiments, the configuration recall module 546 may include recall conditions 546.1 for determining events, trigger states and other operating conditions when previously stored configuration data should be recalled from connected devices. For example, the recall conditions 546.1 may include a set of connection loss events for the peripheral interface, the connection loss event selected from: a power cycle event; a cold boot event; a reset event; a reboot event; a power state change event; and a link error event. In some embodiments, the connection loss event may include an event in which a physical link connection of the storage bus interface 516 is lost and link training and negotiation are expected. The connection loss event may be determined by monitoring general operating states (such as cold boot, power cycle, and power state change) and / or triggering from general operating states that may be identified at the system level before the general operating state must call the link state machine 532.1 to enable initiation of the configuration recall module 546 to recall previously stored configuration data from the connected device before the previously stored configuration data is needed by the storage interface protocol 532. In some embodiments, in response to the recall condition 546.1, the recall message interface 546.2 may initiate a control bus message to recall the previously stored configuration data via the messaging service 566. For example, the recall message interface may include an interface service for initiating the messaging service 566, wherein the service call from the recall message interface 546.2 acts as a sharing trigger event 566.2, the selected backup device (from the configuration backup module 544) completes the shared peer selector parameters, and the configuration data request completes the shared content selector parameters. When messaging service 566 receives one or more responses from connected devices with previously backed-up configuration data, the payloads of those response messages may be returned to recall data manager 546.3. In some embodiments, recall data manager 546.3 may parse the payload data of the responses to determine a previously stored set of configuration data and provide that set of configuration data for use by storage interface protocol 532. For example, recall data manager 546.3 may communicate the set of configuration data (e.g., a set of coefficient values) to coefficient value logic 532.2 and / or store the set of configuration data in registers or other memory locations known to coefficient value logic 532.2 for storing previously used configuration data, if available.

[0068] The peer sharing service 560 may include an interface protocol and a set of functions and parameters for discovering peer storage devices and transmitting and receiving messages with those peer storage devices, such as sharing configuration data of peripheral interfaces supporting the storage interface protocol 532. For example, the peer sharing service 560 may include functionality to utilize low bandwidth communications through the control bus interface 518 using the control bus protocol 540. In some embodiments, the peer sharing service 560 may include management of peer data 520.2 in the non-volatile memory device 520 for storing peer internal operational data and / or may support the transfer of data such as configuration data to other components such as the configuration service 542.

[0069] In some embodiments, the peer sharing service 560 may include a plurality of hardware modules and / or software modules configured to process or manage defined operations of the peer sharing service 560 using the processor 512 and the memory 514. For example, the peer sharing service 560 may include a discovery service 562 and / or a messaging service 566. For example, the discovery service 562 may be configured to discover peer storage devices and / or other connected devices on a shared control bus and store their addresses for use by the messaging service 566. The messaging service 566 may be configured to send and receive one-to-one and / or broadcast messages to and from peer storage devices over the shared control bus.

[0070] The discovery service 562 may include data structures, functions, and interfaces for discovering peers or connected devices on a control bus and determining peer device addresses for use by the peer sharing service 560. For example, during initialization or startup of the storage device 500, the discovery service 562 may broadcast its device identifier or address and receive similar broadcasts from other devices to accumulate device information for other devices on the control bus. In some embodiments, the discovery service 562 may include a peer device registry 562.1 that is configured to store connected device identifiers, control bus addresses, and / or other connected device information to establish and support communications with peer devices. For example, the peer device registry 562.1 may include data tables or other data structures in the memory 514 and / or the device data 520.1 that include multiple connected device entries associated with corresponding control bus addresses. In some embodiments, the discovery service 562 may include a device ID broadcaster configured to determine a storage device identifier and / or a control bus address of the storage device 500 and broadcast a message on the control bus to enable other devices on the control bus to discover the storage device 500. For example, the device ID broadcaster may be triggered during ROM, boot loader, or firmware execution at startup or initialization of the storage device 500 in response to a discovery message from another device and / or another device discovery trigger according to the control bus protocol 540. In some embodiments, the discovery service 562 may include a peer ID receiver configured to monitor communications conducted through the control bus interface 518, identify discovery broadcasts from other storage devices, and add corresponding device identifier entries in the device registry 520.1. For example, the control bus protocol 540 may define the syntax of the broadcast message, such as a broadcast address, a tag, or similar parameters for specifying a discovery broadcast, and a data payload including a storage device identifier, a control bus address, a vendor identifier, etc., and the peer ID receiver may be configured according to a parsed message that satisfies the syntax to extract a peer storage device identifier.

[0071] Messaging service 566 may include data structures, functions, and interfaces for sending and receiving messages to / from peer devices over a control bus. For example, control bus messaging service 566 may implement control bus protocol 540 to send and receive messages with other devices identified by discovery service 562. In some embodiments, messaging service 566 may send packetized data payloads over a control bus using block write and block read commands between buffers in peer storage devices.

[0072] In some embodiments, the messaging service 566 may include a shared peer selector 566.1, which is configured to select a target or destination peer device to receive a message. For example, the shared peer selector 566.1 may include a parameter setting or algorithm for selecting a peer storage device to receive a message. The shared peer selector 566.1 may receive an indication of a target peer device in a call from another component (such as a configuration service 542 and / or a peer backup service 570). In some embodiments, the messaging service 566 may include a shared trigger event 566.2, which is configured to determine when a message is transmitted to one or more peer devices. For example, the shared trigger event 566.2 may include a listener or message handler for an internal call from other components (such as a configuration service 542 and / or a peer backup service 570). In some embodiments, the messaging service 566 may include a shared content selector 566.3, which is configured to select the payload data content of the messaging service 566. For example, the shared content selector 566.3 may receive payload data including an indication of a recall request, configuration data and an indication of a backup request, and / or configuration data and an indication of a recall response from the configuration service 542 and / or the peer backup service 570. In some embodiments, the messaging service 566 may include a shared message formatter 566.4 configured to format a peer message according to the control bus protocol 540 to include the control bus address of the target peer storage device and the selected content in the data payload. For example, the shared message formatter 566.4 may generate or determine a header including a control bus address and a read / write flag, followed by a data payload and / or other parameters, such as a command code and / or an error code, to convey the configuration data.

[0073] In some embodiments, the messaging service 566 may include a peer message monitor 566.5 configured to monitor the control bus interface 518 for messages addressed to the storage device 500 from a peer device (either directly or via a broadcast). For example, the peer message monitor 566.5 may receive peer messages over the control bus initiated by a peer storage device and / or in response to messages sent by the messaging service 566. The peer message monitor 566.5 may parse the received messages to determine whether further action is required, such as storing peer data, passing backup request configuration data to the peer backup service 570, and / or passing recalled configuration data to the configuration service 542.

[0074] The peer backup service 570 may include an interface protocol and a set of functions and parameters for receiving, storing, and returning backup configuration data of other connected devices. For example, the peer backup service 570 may support peer storage devices to back up their configuration data to one or more peer devices, including the storage device 500, via a control bus. In some embodiments, the peer backup service 570 may include a plurality of hardware modules and / or software modules that are configured to use the processor 512 and the memory 514 to process or manage the defined operations of the peer backup service 570. For example, the peer backup service 570 may include a peer backup handler 572, a peer storage manager 574, and / or a peer recall handler 576. For example, the peer backup handler 572 may be configured to receive a backup request from a peer device via a messaging service 566. The peer storage manager 574 may be configured to store configuration data from a peer device for later recall. The peer recall handler 576 may be configured to receive a recall request from a peer device via the messaging service 566 and provide previously stored configuration data back to the peer device via the messaging service 566 .

[0075] The peer backup handler 572 may include data structures, functions, parameters, and / or interfaces for receiving a backup request from a peer device via the messaging service 566. For example, when the messaging service 566 receives a backup request from a connected device, it may parse the message to identify it as a backup request and determine a set of configuration data to store. The messaging service 566 may pass the set of configuration data and the device identifier of the connected device that transmitted the backup request to the peer backup handler 572. The peer backup handler 572 may initiate the peer storage manager 574 to store the received configuration data.

[0076] The peer storage manager 574 may include data structures, functions, parameters and / or interfaces for storing, reading and otherwise managing peer configuration data. For example, the peer storage manager 574 may receive peer configuration data from the peer backup handler 572 and store it in peer data 520.2 in the non-volatile memory 520. In some embodiments, the peer storage manager 574 may use a device registry such as the device registry 562.1 to manage peer configuration data. For example, the peer storage manager 574 may write or update a device registry entry corresponding to the device identifier 574.1 of the source device with the storage location where the configuration data is stored in the peer device configuration data 574.2 and / or the peer data 520.2. When a recall request is received by the peer recall handler 576, the peer storage manager 574 may use the device identifier of the requesting device as an index value to retrieve the device configuration data 574.2 from the storage location where the configuration data is stored in the device registry 562.1 and / or the peer data 520.2. Configuration data read by peer storage manager 574 may be returned to peer recall handler 576 and / or messaging service 566 for return to the requesting peer device.

[0077] The peer recall handler 576 may include data structures, functions, parameters, and / or interfaces for receiving and fulfilling recall requests from connected devices that have previously backed up their configuration data to the storage device 500. For example, the peer recall handler 576 may receive a peer recall request received by the messaging service 566, retrieve the requested configuration data using the peer storage manager 574, and initiate the messaging service 566 to return the configuration data to the requesting peer device. The messaging service 566 may transmit the recall request (including the device identifier of the requesting device) to the peer recall handler 576, and the peer recall handler 576 may initiate the peer storage manager 574 to locate and return the requested configuration data using the device identifier. In response to receiving the previously stored configuration data from the peer storage manager 574, the peer recall handler 576 may instruct the messaging service 566 to return the configuration data in a response message to the recall request.

[0078] like Figure 6 As shown, an example method for accelerating link training using stored coefficient values ​​(i.e., according to Figure 6 The storage device 500 is operated according to the method 600 illustrated in blocks 610 to 634 in FIG. 600. The block indicated by 602 illustrates using the received coefficient values ​​to accelerate link training. The block indicated by 604 illustrates backing up the trained coefficient values ​​to the peer device via the control bus, and the block indicated by 606 illustrates the peer device receiving and storing the coefficient values ​​to be returned for future operations of the method 600.

[0079] At block 610, the connected device returns to operation. For example, the connected device may be powered on after a previous operation cycle and powered off if coefficient values ​​were previously determined but lost due to the power off.

[0080] At block 612, coefficient values ​​may be requested from a peer device. For example, where a connected device has previously stored coefficient values ​​to another connected device, the device may request recall of the coefficient values ​​using control bus messaging.

[0081] At block 614, link training is initiated. For example, as part of a cold boot or power state change, the link state machine may monitor the return of the physical and data links of the peripheral interface, which may include physical link training and negotiation between the connected device and the upstream system.

[0082] At block 616, a determination may be made as to whether previously stored coefficient values ​​have been received. For example, the link state machine may include coefficient value logic for checking whether previously stored coefficient values ​​are available from a memory location or internal call. If previously stored coefficient values ​​have not been received, the method 600 may continue to block 618 for standard link negotiation. If previously stored coefficients have been received, the method 600 may proceed to block 626 to attempt to accelerate link training using the received coefficient values.

[0083] At block 618, link training may be negotiated using default coefficient values. For example, the connected device may select a default set of preset coefficient values ​​to initiate negotiation, transmit the default set of preset coefficient values ​​to the upstream system via the peripheral interface, and check the resulting bit error rate.

[0084] At block 620, optimal coefficient values ​​may be determined. For example, the coefficient value logic may iterate through a failing set of coefficient values ​​until a desired bit error threshold is achieved, and those coefficient values ​​are selected as the optimal coefficient values ​​for this link.

[0085] At block 622, link training may be completed. For example, the optimal coefficient values ​​may enable a target data rate to meet a desired bit error threshold and establish a physical and data link to support application-level communications over the peripheral interface.

[0086] At block 624, the stored coefficient values ​​may be received from the peer device. For example, another connected device that previously stored a set of optimal coefficient values ​​for the requesting device may return the stored coefficient values ​​using control bus messaging.

[0087] At block 626, link training may be negotiated using the received coefficient values. For example, the device may provide the received coefficient values ​​to coefficient value logic of a link state machine for use in link training, and the received coefficient values ​​may be used as initial values ​​for negotiation.

[0088] At block 628, the link error rate may be verified. For example, after the received coefficient values ​​are transmitted to the upstream device, the downstream device may check the bit error rate of the test data sequence transmitted using those coefficients to verify whether the expected bit error threshold is met. If so, link training (or at least some phase of link training) may be completed at block 622. If not, the method 600 may proceed to block 620 and iterate through the coefficient values ​​using a standard link negotiation process.

[0089] At block 630, the coefficient values ​​may be communicated to one or more peer devices. For example, based on successful link training and establishing application-level communications through a peripheral interface, a device may initiate a backup of current optimal coefficient values ​​for future use in method 600 and communicate the optimal coefficient values ​​to one or more connected devices using control bus messaging.

[0090] At block 632, one or more peer devices may receive the coefficient values. For example, each connected device to which the optimal coefficient values ​​are communicated may receive a configuration data backup request in one or more control bus messages.

[0091] At block 634, one or more peer devices may store the coefficient values. For example, each connected device that receives the configuration data backup request may store the coefficient values ​​associated with the device identifier of the transmitting device and return those stored coefficient values ​​in future iterations through method 600 upon receiving a recall request message to support block 624.

[0092] like Figure 7 As shown, an example method for accelerating link training using stored configuration data (i.e., according to Figure 7 700) to operate the storage device 500. The method 700 may be implemented over a first operating cycle 702, followed by a second operating cycle 704, the two operating cycles being generally separated by at least one connection loss event. The first operating cycle 702 may be an initial operating cycle after initialization of the device, wherein a reliable physical and data link is established through a peripheral interface prior to the second operating cycle.

[0093] At block 710, a device is connected to the control bus. For example, a data storage device may be physically and electrically connected to the control bus through a control bus interface.

[0094] At block 712, communications over the control bus may be established. For example, the data storage device may use a control bus protocol to establish communications with one or more connected devices over the control bus.

[0095] At block 714, the device is connected to the peripheral bus. For example, the data storage device may be physically and electrically connected to the peripheral interface bus via a storage bus interface.

[0096] At block 716, a connection to the host system may be initiated via the peripheral interface bus. For example, a link state machine according to the peripheral interface protocol may initiate an electrical connection and proceed through various states to attempt to establish application-level communications via the peripheral interface bus.

[0097] At block 718, link configuration data may be determined. For example, the link state machine may negotiate link training between the data storage device and the host system to determine various configuration parameters, including coefficient values ​​for optimizing the physical and data links across the peripheral interface bus.

[0098] At block 720, communications may be established over the peripheral interface bus. For example, the state machine may implement a desired link state having a target data rate and meeting a desired bit error rate threshold to support reliable data, transaction, and application level communications across the peripheral interface bus.

[0099] At block 722, the configuration data may be transmitted to other devices. For example, the data storage device may communicate with other connected devices over a control bus and use control bus messaging to back up configuration data (such as a set of optimal coefficient values) to one or more of those connected devices.

[0100] At block 724, after the connection loss event, a connection loss event is determined. For example, the data storage device may determine that it is returning to normal operation from a cold boot, power cycle, power state change, or other interface error or interruption and anticipates that link training for the peripheral interface is necessary to return to normal operation.

[0101] At block 726, a request message may be transmitted to at least one other connected device. For example, the data storage device may transmit a recall request message to one or more peer data storage devices, the host BIOS, and / or the BMC over the control bus prior to the restoration of the peripheral interface and the initiation of link training.

[0102] At block 728, a response message including the configuration data may be received. For example, the data storage device may receive a response message from one or more connected devices via a control bus, with the configuration data in a response message payload.

[0103] At block 730, a connection may be initiated to the host system via the peripheral bus. For example, the link state machine may initialize the connection process as described above with respect to block 716, but may now initiate link training negotiations using received configuration data (such as previously stored optimal coefficient values).

[0104] The received configuration data may be used to reestablish communications over the peripheral interface bus at block 732. For example, use of the received configuration data may speed up the link negotiation phase and allow the data storage device to more quickly reestablish physical and data links to support transaction and application level communications.

[0105] like Figure 8 As shown, the configuration data can be stored in the connected device according to the example method for storing the configuration data through the control bus (ie, according to the example method of Figure 8 810 to 820 in the method 800) to operate the storage device 500.

[0106] At block 810, connected devices may be discovered. For example, a control bus protocol may define a discovery process for devices connected to the control bus, and the discovery process may be used to determine other connected devices on the control bus and their corresponding device identifiers and control bus addresses.

[0107] At block 812, communications may be established over the control bus. For example, based on the discovered connected devices, the device may use a messaging protocol for direct messaging and / or broadcast messaging over the control bus.

[0108] At block 814, configuration data may be determined. For example, the device may determine configuration data for another interface, such as a high throughput peripheral interface, that includes optimal parameters for reestablishing application-level communications over the peripheral interface.

[0109] At block 816, the configuration data may be broadcast to the connected devices. For example, the device may broadcast a backup message with the configuration data as a payload to all connected devices on the control bus.

[0110] At block 818, target connected devices may be determined.For example, the device may select one or more connected devices to receive configuration data via direct messaging over the control bus.

[0111] At block 820, the configuration data may be transmitted to the target connected devices. For example, the device may transmit a backup message with the configuration data as a payload to one or more target connected devices.

[0112] like Fig. 9 As shown, the example method for receiving and returning configuration data via the control bus as a connected device (i.e., according to Fig. 9 912 to 928 in the method 900) to operate the storage device 500. Blocks 912 to 916 illustrate example connected devices that can provide configuration data backup via the control bus in various embodiments.

[0113] Configuration data may be received at a host BIOS at block 912. For example, a host system may include a BIOS that interfaces with a control bus for power and component management and may be used to receive and transmit configuration data for other connected devices early in the boot process before establishing communications over a peripheral interface.

[0114] Configuration data may be received at the BMC at block 914. For example, the BMC may interface with a control bus for power and component management and may be used to receive and transmit configuration data for other connected devices early in the boot process and / or independent of the power state of other connected devices.

[0115] At block 916, configuration data may be received at the peer device. For example, multiple peer devices may be connected to the control bus and may be available early in the boot process to receive and transmit configuration data for the device that backs up the configuration data and / or operate in a different power state than the device that backs up the configuration data to receive and transmit configuration data for the device that backs up the configuration data.

[0116] At block 918, a source device identifier may be determined. For example, the backup request message received at blocks 912 through 916 may include a source device identifier for the device backing up the configuration data, such as in header data of the message that may be parsed by the receiving device.

[0117] At block 920, configuration data values ​​may be determined. For example, the backup request message received at blocks 912 through 916 may include configuration data in a payload, and the receiving device may parse the payload data to determine configuration data values, such as a set of coefficient values.

[0118] At block 922, a connected device entry may be determined. For example, based on the source device identifier, the receiving device may determine a connected device entry in a connected device registry in the receiving device.

[0119] At block 924, the configuration data may be stored in the connected device registry. For example, the receiving device may add the configuration data value to the connected device entry determined at block 922 and / or add / update a pointer in the connected device entry to a storage location for the configuration data in the receiving device's local memory.

[0120] At block 926, a recall request may be received following a connection loss event of the source device. For example, the receiving device may receive a recall request message specifying a source device identifier and configuration data to be recalled via a control bus.

[0121] The configuration data may be communicated back to the source device at block 928. For example, the receiving device may use the connected device entry for the source device identifier to determine previously stored configuration data and communicate the configuration data in a payload of a response message to the recall request message over the control bus.

[0122] Although at least one exemplary embodiment has been presented in the foregoing detailed description of the present technology, it should be understood that a large number of variations are possible. It should also be understood that one or more exemplary embodiments are examples and are not intended to limit the scope, applicability, or configuration of the technology in any way. On the contrary, the foregoing detailed description will provide those skilled in the art with a convenient guideline for implementing the exemplary embodiments of the present technology, and it should be understood that various modifications may be made to the functions and / or arrangements of the elements described in the exemplary embodiments without departing from the scope of the technology as set forth in the attached claims and their legal equivalents.

[0123] As will be appreciated by those of ordinary skill in the art, various aspects of the present technology may be embodied as systems, methods, or computer program products. Therefore, some aspects of the present technology may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or a combination of hardware and software aspects, which may generally be referred to herein as circuits, modules, systems, and / or networks. In addition, various aspects of the present technology may take the form of a computer program product embodied in one or more computer-readable media, including computer-readable program codes embodied thereon.

[0124] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or physical computer-readable storage media. For example, physical computer-readable storage media can be, but are not limited to, electronic, magnetic, optical, crystal, polymer, electromagnetic, infrared or semiconductor systems, devices or equipment, etc., or any suitable combination of the foregoing. Non-limiting examples of physical computer-readable storage media can include, but are not limited to, electrical connections including one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), flash memories, optical fibers, compact disc read-only memories (CD-ROM), optical processors, magnetic processors, etc., or any suitable combination of the foregoing. In the context of this document, computer-readable storage media can be any tangible medium that can contain or store programs or data for use or in combination with instruction execution systems, devices, and / or equipment.

[0125] The computer code embodied on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, fiber optic cable, radio frequency (RF), etc., or any suitable combination of the foregoing. The computer code used to perform the operations of various aspects of the present technology can be written in any static language, such as the C programming language or other similar programming languages. The computer code can be executed entirely on the user's computing device, partially on the user's computing device, as a separate software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the latter case, the remote computing device can be connected to the user's computing device through any type of network or communication system, including but not limited to, a local area network (LAN) or wide area network (WAN), a converged network or connection can be established to an external computer (e.g., through the Internet using an Internet service provider).

[0126] Aspects of the present technology may be described above with reference to flowchart illustrations and / or block diagrams of methods, devices, systems, and computer program products. It should be understood that each block of the flowchart illustration and / or block diagram and the combination of blocks in the flowchart illustration and / or block diagram may be implemented by computer program instructions. These computer program instructions may be provided to a processing device (processor) of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, so that instructions executed by the processing device or other programmable data processing apparatus may be used to create an apparatus for implementing the operations / actions specified in the blocks of the flowchart and / or block diagram.

[0127] Some computer program instructions may also be stored in a computer-readable medium that may instruct a computer, other programmable data processing apparatus, or other device to operate in a particular manner, such that the instructions are stored in the computer-readable medium to produce an article of manufacture including instructions for implementing the operations / actions specified in the flowchart and / or in the blocks of the block diagram. Some computer program instructions may also be loaded onto a computing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing device, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executed by the computer or other programmable apparatus provide one or more processes for implementing the operations / actions specified in the blocks of the flowchart and / or block diagram.

[0128] The flowcharts and / or block diagrams in the above figures may illustrate the possible specific implementation architecture, functions and / or operations of the devices, systems, methods and / or computer program products according to various aspects of the present technology. In this regard, the blocks in the flowchart or block diagram may represent modules, fragments or parts of the code, which may include one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative aspects, some functions shown in the blocks may not occur in the order shown in the figure. For example, in fact, the two blocks shown in succession may be executed substantially simultaneously, or these blocks may sometimes be executed in the opposite order, depending on the operations involved. It should also be noted that the blocks illustrated in the block diagram and / or flowchart or the combination of blocks in the block diagram and / or flowchart can be implemented by a system based on dedicated hardware that can perform one or more specified operations or actions, or a combination of dedicated hardware and computer instructions.

[0129] While one or more aspects of the present technology have been described and discussed in detail, those of ordinary skill in the art will appreciate that modifications and / or adaptations may be made to the various aspects without departing from the scope of the present technology as set forth in the appended claims.

Claims

1. A system comprising: A first device, the first device comprising: a peripheral interface configured to connect to a host system using a peripheral interface protocol; a control bus interface configured to connect to a control bus using a control bus protocol, wherein at least a second device is configured to connect to the control bus; a configuration service configured to receive configuration data for the peripheral interface protocol from the second device via the control bus interface; and A link state machine is configured to use the configuration data received from the second device to reestablish communications to the host system through the peripheral interface.

2. The system of claim 1, wherein: The first device also includes a control bus messaging service, the control bus messaging service being configured to: establishing communication with the second device via the control bus; transmitting, during a first operating period, to the second device a first peer message including the configuration data; as well as transmitting a second peer message to the second device during a second operating period to request the configuration data; The configuration service is further configured to determine the configuration data based on establishing communications to the host system via the peripheral interface during the first operating cycle; The second operating cycle is subsequent to the first operating cycle; and Reestablishing communications through the peripheral interface is during the second operating cycle.

3. The system of claim 2, wherein the second operation cycle is in response to a connection loss event of the peripheral interface, the connection loss event being selected from: Power cycle events; Cold boot events; Reboot events; Reset event; Power state change events; and Link error event.

4. The system of claim 1, wherein: The first device also includes a control bus messaging service, the control bus messaging service being configured to: establishing communication with a plurality of connected devices including the second device; as well as broadcasting a first peer message including the configuration data to the plurality of connected devices; and Each connected device of the plurality of connected devices is configured to store the configuration data.

5. The system of claim 4, wherein: The first device and each connected device of the plurality of connected devices are configured to store a connected device registry; Each connected device registry includes a connected device entry corresponding to each of the plurality of connected devices and the first device; and Each connected device entry includes: Connected device identifier; and Corresponding configuration data.

6. The system of claim 1, wherein: The configuration data includes a set of coefficient values ​​for physical link training; and The link state machine is further configured to use the configuration data to negotiate physical link training between the first device and the host system to reestablish communications over the peripheral interface according to the peripheral interface protocol.

7. The system of claim 6, wherein the set of coefficient values ​​comprises a coefficient preset of a physical link filter value selected from a plurality of coefficient presets defined by the peripheral interface protocol.

8. The system of claim 1, further comprising: A plurality of data storage devices, the plurality of data storage devices being configured to: communicating with storage of the host system via a peripheral interface bus using the peripheral interface protocol; as well as communicating with a peer data storage device of each of the plurality of data storage devices via the control bus using the control bus protocol, wherein: The first device is a first data storage device of the plurality of data storage devices; and The second device is a second data storage device among the plurality of data storage devices.

9. The system of claim 1, wherein: The first device is a data storage device; and The second device is selected from: The host system comprises: a host control bus interface configured for communicating with the data storage device via the control bus using the control bus protocol; and a basic input-output system configured to store the configuration data from a previous operation cycle of the data storage device; and A baseboard management controller, the baseboard management controller comprising: a management control bus interface configured to communicate with the data storage device via the control bus using the control bus protocol; and A management memory is configured to store the configuration data from a previous operation cycle of the data storage device.

10. The system of claim 1, further comprising: A plurality of data storage devices, the plurality of data storage devices being configured to: communicating with storage of the host system via a peripheral interface bus using the peripheral interface protocol; as well as communicating with a control bus of the host system via the control bus using the control bus protocol, wherein control bus communications are established prior to storage communications during initialization of the data storage device; The host system comprises: a host control bus interface configured to communicate with the plurality of data storage devices via the control bus using the control bus protocol; and a memory configured to store, for each of the plurality of data storage devices, configuration data from a previous operation cycle of that data storage device, wherein: The first device is a data storage device from the plurality of data storage devices; and The second device is the host system.

11. A computer-implemented method comprising: connecting, by a first device, to a control bus using a control bus protocol, wherein at least a second device is configured to connect to the control bus; receiving configuration data for a peripheral interface protocol from the second device via the control bus; Initiating, by the first device, a connection to a host system via a peripheral interface using the peripheral interface protocol; as well as Communications are reestablished to the host system through the peripheral interface using the configuration data received from the second device.

12. The computer-implemented method of claim 11 , further comprising: determining the configuration data based on establishing communications to the host system through the peripheral interface during a first operating cycle; The first device establishes communication with the second device via the control bus; transmitting, from the first device to the second device during the first operating period, a first peer message including the configuration data; as well as transmitting a second peer message to the second device during a second operating period to request the configuration data, wherein: The second operating cycle is subsequent to the first operating cycle; and Reestablishing communications through the peripheral interface is during the second operating cycle.

13. The computer-implemented method of claim 12, further comprising: A connection loss event of the peripheral interface is determined by the first device, wherein: The connection loss event is selected from: Power cycle events; Cold boot events; Reboot events; Reset event; Power state change events; and Link error events; and The second operation cycle is responsive to the connection loss event.

14. The computer-implemented method of claim 11 , further comprising: establishing, by the first device, communication with a plurality of connected devices including the second device; broadcasting, by the first device, a first peer message including the configuration data to the plurality of connected devices; as well as The configuration data is stored by each connected device of the plurality of connected devices.

15. The computer-implemented method of claim 14, further comprising: A connected device registry is stored by the first device and each of the plurality of connected devices, wherein: Each connected device registry includes a connected device entry corresponding to each of the plurality of connected devices and the first device; and Each connected device entry includes: The connected device identifier; and Corresponding configuration data.

16. The computer-implemented method of claim 11, further comprising: The configuration data is used to negotiate physical link training between the first device and the host system according to the peripheral interface protocol to reestablish communications over the peripheral interface, wherein the configuration data includes a set of coefficient values ​​for physical link training.

17. The computer-implemented method of claim 16, wherein the set of coefficient values ​​comprises a coefficient preset of a physical link filter value selected from a plurality of coefficient presets defined by the peripheral interface protocol.

18. The computer-implemented method of claim 11, wherein: The first device is a first data storage device among a plurality of data storage devices, and the first data storage device is configured to: peer communications via said control bus; as well as Communicate with the storage of the host system via a peripheral interface bus; and The second device is a second data storage device among the plurality of data storage devices.

19. The computer-implemented method of claim 11, further comprising: receiving, by the host system during a first operation cycle, the configuration data from the first device via the control bus; storing the configuration data in a memory of the host system during the first operating cycle; as well as The configuration data is transmitted by the host system to the first device over the control bus during a second operating cycle prior to reestablishing communications over the peripheral interface.

20. A storage system, the storage system comprising a plurality of data storage devices, wherein each data storage device comprises: a storage interface configured to connect to a host system; a control bus interface configured to connect to a control bus; a storage medium configured to store host data; means for receiving configuration data for a storage interface protocol from another data storage device of the plurality of data storage devices via the control bus interface; means for initiating a connection to the host system through the storage interface using the storage interface protocol; and means for reestablishing communications to the host system via the storage interface using the configuration data received from another data storage device of the plurality of data storage devices.

Images