Asymmetric PCIe bus design methods, devices, bus devices and systems

By designing an asymmetric PCIe bus and modifying the PCIe interface module and link training state machine logic, the problem of unequal read and write bandwidth of the PCIe bus was solved, achieving efficient data transmission and improved system performance in AI computing.

CN116680224BActive Publication Date: 2026-11-17INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310530559.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2026-11-17
Estimated Expiration
2043-05-11

AI Technical Summary

Technical Problem

In AI computing, the asymmetry between read and write bandwidths on the PCIe bus makes PCIe bandwidth a key bottleneck, especially when a device loads a large amount of data and then sends back a small amount of data. Existing technologies struggle to effectively address this issue.

Method used

Design an asymmetric PCIe bus by modifying the physical lines and link training state machine detection state logic of the PCIe interface modules on the host and device sides. This allows each channel in the transmission channel from the host to the device to be configured with multiple transmitters corresponding to one receiver, and information is returned through the receivers in the same channel, thus achieving asymmetric bandwidth requirements.

Benefits of technology

It enables efficient data transmission in AI computing, improves the flexibility of server configuration, solves the PCIe bandwidth bottleneck, improves the overall system performance, and has a reliable design principle and simple structure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116680224B_ABST
    Figure CN116680224B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of PCIe bus design technology, specifically providing an asymmetric PCIe bus design method, apparatus, bus device, and system. The method includes the following steps: determining that the connection between the host and device ends has asymmetric bandwidth requirements; modifying the physical wiring of the PCIe interface modules of the host and device ends based on the asymmetric bandwidth requirements, so that each channel in the transmission channel from the host to the device end is configured with multiple transmitters corresponding to one receiver; modifying the logic of the detection state of the link training state machine between the host and device ends, so that information received by multiple transmitters is returned to the host end through the receiver in the same channel. This enables normal data transmission when connecting to an asymmetric PCIe bus, improves server configuration flexibility, solves the PCIe bandwidth problem in AI computing, and improves the overall system performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of PCIe bus design technology, and more specifically to asymmetric PCIe bus design methods, apparatus, bus devices, and systems. Background Technology

[0002] The PCIe bus is a high-speed serial bus used to connect various computer peripherals, such as graphics cards, network cards, sound cards, and storage devices. The PCIe bus operates similarly to network transmission, with data transmitted through multiple differential signal lines, each transmitting one bit of binary data. PCIe bus speed is typically measured in bytes per second (GB / s); for example, PCIe 3.0 is 8 GB / s. There are several versions of the PCIe bus, each with different speeds and bandwidths. The latest version is PCIe 5.0, which is twice as fast as PCIe 4.0. The PCIe bus also has multiple channels, each with a certain bandwidth, allowing multiple data streams to be transmitted simultaneously. The PCIe bus is a high-speed, reliable, and flexible bus suitable for connecting various peripherals, providing efficient data transmission and performance.

[0003] Deep learning is an artificial intelligence technology. Key technologies behind deep learning include backpropagation algorithms, convolutional neural networks, recurrent neural networks, generative adversarial networks, and more. Deep learning also requires massive amounts of data and computing resources to train and optimize models; therefore, large-scale deep learning applications typically require high-performance computing and large-scale data storage and processing capabilities. In-memory computing (CIM) is a processor architecture that integrates memory and computing functions. Unlike traditional computer architectures, CIM allows computational operations to be performed directly in memory, significantly improving computational efficiency and energy efficiency. The core idea of ​​CIM is to combine memory and processor, thus avoiding the bottleneck of frequent data transfer between memory and processor in traditional computers. Specifically, CIM achieves localization of computation and storage by integrating processing units in memory. It can directly perform computationally intensive operations such as matrix multiplication and convolution in memory, thereby greatly reducing data transfer and latency, and improving computational efficiency and energy efficiency.

[0004] CIM (Computer Integrated Processing) technology has been widely applied in fields such as artificial intelligence accelerators, image processors, and computer vision processors. Compared with traditional processors, CIM can achieve higher parallelism, lower power consumption, and higher performance. Furthermore, CIM can further improve computational efficiency and energy efficiency through the integration of hardware accelerators and software optimization. During AI model training, the read and write bandwidths of the PCIe bus are not equal; devices load large amounts of data and then send back small amounts of data after obtaining the calculation results. Therefore, PCIe bandwidth is a key bottleneck in AI computing. How to design an asymmetric PCIe bus topology is the technical problem addressed in this application. Summary of the Invention

[0005] When training AI models, the read and write bandwidth of the PCIe bus is not equal. The device will load a large amount of data and send back a small amount of data after obtaining the calculation results. The bandwidth of PCIe will be the key bottleneck of AI computing. In view of this, the present invention provides an asymmetric PCIe bus design method, device, bus device and system.

[0006] In a first aspect, the present invention provides an asymmetric PCIe bus design method, wherein the asymmetric PCIe bus is used to connect a host end and a device end, and both the host end and the device end are provided with PCIe interface modules. The method includes the following steps:

[0007] It is determined that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, the physical lines of the PCIe interface modules of the host and the device are modified so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver, or each channel is configured with multiple receivers corresponding to one transmitter.

[0008] The logic for detecting the state of the training state machine between the host and device ends is modified so that information received by multiple sending ends is returned to the host end through the receiving end in the same channel, or information received by the sending end is distributed to each receiving end in the same channel and returned to the host end.

[0009] As a preferred embodiment of the technical solution of the present invention, the step of modifying the detection state logic of the host-side and device-side link training state machine includes:

[0010] The sender's number is set in the detection sequence sent by the sender, and detection sequences sent by multiple senders are received by the receiver of the same channel.

[0011] After receiving the detection sequence, the receiving end identifies the corresponding sending end by the number.

[0012] As a preferred embodiment of the technical solution of the present invention, the step of setting the sender's number in the detection sequence sent by the sender includes:

[0013] Edit the number used to identify the sender in the start code of the detection sequence sent by the sender, so that the total length of the detection sequence remains unchanged.

[0014] As a preferred embodiment of the technical solution of the present invention, the logic of detecting the state of the link training state machine between the host end and the device end is modified so that the information received by multiple sending ends is returned to the host end through the receiving end in the same channel. This is followed by:

[0015] Perform link training on one or more channels connecting the host to the device;

[0016] During link training, all configured senders in the detection channel transmit data, and all configured receivers receive data and identify the corresponding senders.

[0017] As a preferred embodiment of the technical solution of the present invention, the detection state of the state machine includes detecting a static state and detecting an active state, and the step of performing link training on one or more channels connected from the host to the device includes:

[0018] After the high-speed serial extended bus device on the device side enters the detection active state, the transmitting end sends a detection sequence with the transmitting end number to detect whether the receiving end is normal.

[0019] If all the detection sequences with transmitter numbers sent by all transmitters of all channels are received normally and all transmitters are identified by their numbers, then the receiver detection is normal and the high-speed serial extended bus device enters the link training state.

[0020] If all the detection sequences with transmitter numbers sent by all transmitters on all channels are not received normally, the receiver detects an anomaly, and the high-speed serial extended bus device returns to the detection standby state.

[0021] As a preferred embodiment of the technical solution of the present invention, the method further includes:

[0022] If some channels successfully receive the detection sequence with the sender's number and identify the sender by the number, after a set time interval, the senders of all channels that did not receive the detection sequence successfully will resend the detection sequence with the sender's number.

[0023] If the resent test result is the same as the previous test result;

[0024] Set all undetected channels to the electrically idle state and enter the link training state;

[0025] If the resent detection result is different from the previous detection result, the detection will return to a standstill state.

[0026] As a preferred embodiment of the technical solution of the present invention, the method further includes:

[0027] When a high-speed serial expansion bus device is in a detection idle state for more than a set time threshold, or when any channel in the high-speed serial expansion bus is detected to have exited the electrical idle state, the high-speed serial expansion bus device enters the detection active state.

[0028] As a preferred embodiment of the technical solution of the present invention, the asymmetric bandwidth requirement includes a ratio of 3:1 between the transmitting end and the receiving end in the bus channel.

[0029] Secondly, the present invention also provides an asymmetric PCIe bus design device, wherein the asymmetric PCIe bus is used to connect the host end and the device end, and both the host end and the device end are provided with PCIe interface modules. The device includes a channel design module and a state machine setting module.

[0030] The channel design module is used to determine that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, it modifies the physical lines of the PCIe interface modules of the host and the device so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver, or each channel is configured with multiple receivers corresponding to one transmitter.

[0031] The state machine setting module is used to modify the detection state logic of the training state machine between the host and device ends, so that information received by multiple sending ends is returned to the host end through the receiving end in the same channel, or information received by the sending end is distributed to each receiving end in the same channel and returned to the host end.

[0032] As a preferred embodiment of the technical solution of the present invention, the state machine setting module includes a sequence setting unit and an identification unit;

[0033] The sequence setting unit is used to set the sender's number in the detection sequence sent by the sender. Detection sequences sent by multiple senders are received by the receiver of the same channel. Specifically, it is used to edit the number used to identify the sender in the start code of the detection sequence sent by the sender, so that the total length of the detection sequence remains unchanged.

[0034] The identification unit is used by the receiving end to identify the sending end by number after receiving the detection sequence.

[0035] As a preferred embodiment of the technical solution of the present invention, the device further includes a link training execution module, which is used to perform link training on one or more channels connected from the host to the device; during the link training, it detects all configured sending ends in the channels to send data, and configures receiving ends to receive data and identifies the corresponding sending ends.

[0036] The state machine's detection states include a detection idle state and a detection active state. After the high-speed serial extended bus device enters the detection active state, the link training execution module performs the following process: The transmitting end sends a detection sequence with a transmitting end number to check if the receiving end is normal. If all transmitting ends of all channels send detection sequences with transmitting end numbers that are normally received, and all transmitting ends are identified by their numbers, then the receiving end's detection is normal, and the high-speed serial extended bus device enters the link training state. If all transmitting ends of all channels fail to receive detection sequences with transmitting end numbers that are normally received, then the receiving end's detection is abnormal, and the high-speed serial extended bus device returns to the detection idle state. If some channels normally receive detection sequences with transmitting end numbers that are normally received, and the transmitting end is identified by its number, after a set time interval, all transmitting ends of channels that did not receive the sequence normally resend the detection sequence with the transmitting end number. If the resent detection result is the same as the previous detection result, all channels that failed detection are set to the electrical idle state and enter the link training state. If the resent detection result is different from the previous detection result, the system returns to the detection idle state. When a high-speed serial expansion bus device is in a detection idle state for more than a set time threshold, or when it is detected that any channel in the high-speed serial expansion bus has exited the electrical idle state, the high-speed serial expansion bus device enters the detection active state.

[0037] Thirdly, the present invention also provides an asymmetric PCIe bus device, including a host processor; the host processor includes logic circuitry for the following operations:

[0038] Send a command signal to a device that communicates with a host processor via a PCIe link to indicate the ratio of the transmitting end to the receiving end in the same channel; wherein the ratio is not equal to 1;

[0039] After receiving the instruction from the device that it can support the asymmetric configuration ratio, the host processor dynamically configures the channel to either sender or receiver based on the ratio of the sender to receiver, and connects the communication signal to the device through the configured channel.

[0040] As a preferred embodiment of the present invention, the host processor includes logic circuitry for the following operations:

[0041] The mode of receiving data from a device through a single receiver; and the mode of sending data to a device through multiple transmitters.

[0042] As a preferred embodiment of the present invention, the host processor includes logic circuitry for the following operations:

[0043] The data transmission mode involves sending data to the device through a single transmitter and receiving data from the device through multiple receivers.

[0044] As a preferred embodiment of the present invention, the device further includes a PCIe interface module, which includes logic circuitry for configuring the physical layer for asymmetric transmitter and receiver configuration, and logic circuitry for processing signals of the physical layer configured for asymmetric transmitter and receiver.

[0045] As a preferred embodiment of the present invention, the PCIe interface module includes a logic circuit for splitting data packets according to the ratio of the sending end to the receiving end in the same channel.

[0046] Fourthly, the present invention also provides an asymmetric PCIe bus system, including a host processor and peripheral devices, wherein the peripheral devices include logic circuits for the following operations:

[0047] Identify multiple devices coupled to the host processor via PCIe links;

[0048] The system identifies the read / write ratio of the host processor and the device based on the application of the device and the host processor, and provides the ratio to the host processor.

[0049] The host processor includes logic circuitry for the following operations:

[0050] Send a command signal to a device that communicates with the host processor via a PCIe link to indicate the ratio of the transmitting end to the receiving end in the same channel; wherein the ratio is not equal to 1; the ratio is the ratio provided by the peripheral device to the host processor;

[0051] After receiving the instruction from the device that it can support the asymmetric configuration ratio, the host processor dynamically configures the channel to be either a transmitter or a receiver based on the ratio of the transmitter to the receiver, and connects the communication signal to the device through the configured channel.

[0052] As a preferred embodiment of the present invention, the host processor includes logic circuitry for the following operations:

[0053] The mode of receiving data from a device through a single receiver; and the mode of sending data to a device through multiple transmitters.

[0054] As a preferred embodiment of the present invention, the host processor includes logic circuitry for the following operations:

[0055] The data transmission mode involves sending data to the device through a single transmitter and receiving data from the device through multiple receivers.

[0056] As a preferred embodiment of the present invention, the PCIe link includes one of a PCIe x4 slot, a PCIe x12 slot, a PCIe x16 slot, or a PCIe x32 slot that interconnects the host processor with the device.

[0057] As can be seen from the above technical solutions, the present invention has the following advantages:

[0058] When connecting to an asymmetric PCIe bus, data transmission can be performed normally, greatly improving the flexibility of server configuration. The asymmetric PCIe bus design increases the number of transmitters and reduces the number of receivers, and the state machine is modified to solve the PCIe bandwidth problem in AI computing and improve the overall system performance.

[0059] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects.

[0060] Therefore, it is evident that the present invention has outstanding substantive features and significant progress compared with the prior art, and the beneficial effects of its implementation are also obvious. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0063] Figure 2 This is a schematic diagram illustrating the connection between a CPU and a PCIe device via an asymmetric PCIe bus according to an embodiment of the present invention.

[0064] Figure 3 This is an LTSSM state machine diagram. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0066] It should be noted that PCIe, or Peripheral Component Interconnect Express, is a high-speed serial computer expansion bus standard.

[0067] CIM stands for Compute-In-Memory Processor.

[0068] AI stands for Artificial Intelligence.

[0069] LTSSM, or Link Training and Status State Machine.

[0070] This invention provides a method for designing an asymmetric PCIe bus server. The asymmetric PCIe bus is used to connect the host and the device. Both the host and the device are equipped with PCIe interface modules. The method includes the following steps:

[0071] Step 1: Determine that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, modify the physical lines of the PCIe interface modules of the host and the device so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver.

[0072] Step 2: Modify the detection state logic of the link training state machine between the host and device ends so that information received by multiple sending ends is returned to the host end through the receiving end in the same channel.

[0073] This application expands the flexibility of hardware configuration for asymmetric bus connections based on existing resources, providing a more cost-effective option for scenarios requiring large-volume unidirectional data transfer, such as deep learning computations or connecting in-memory computing chips. This application requires hardware support on both the host and device sides, modifying the link training state machine settings. Due to changes in the application's signals, and modifications to circuitry and components on the PCB board to optimize signal quality, this allows for asymmetric data transmission lines, more efficient use of PCB board space resources, and more flexible support for multiple device connections.

[0074] This invention provides an asymmetric PCIe bus design method. The asymmetric PCIe bus is used to connect the host and device sides, both of which are equipped with PCIe interface modules, such as... Figure 1 As shown, the method includes the following steps:

[0075] Step 1: Determine that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, modify the physical lines of the PCIe interface modules of the host and the device so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver.

[0076] That is, on the host side, design the corresponding number of channels to the high-speed serial expansion bus device side according to the type of the connected high-speed serial expansion bus device;

[0077] Step 2: Modify the detection state logic of the link training state machine between the host and device ends so that information received by multiple sending ends is returned to the host end through the receiving end in the same channel.

[0078] In this step, the logic of the detection state in the training state machine of the host and device links is modified so that the information received by the sending end is returned to the host end through the receiving end in the same channel. Specifically, a sender number is set in the detection sequence sent by the sending end, and detection sequences sent by multiple sending ends are received by the receiving end in the same channel; after receiving the detection sequence, the receiving end identifies the sending end by the number. In fact, the number used to identify the sending end is edited in the start code of the detection sequence sent by the sending end, while keeping the total length of the detection sequence unchanged.

[0079] Step 3: Perform link training on one or more channels connected from the host to the device; during link training, detect all configured senders in the channels to send data, and configure receivers to receive data and identify the corresponding senders.

[0080] This application expands the flexibility of hardware configuration on the server motherboard based on existing resources, providing a more cost-effective option for scenarios requiring large-volume unidirectional data transmission, such as deep learning computing or connecting in-memory computing chip devices. This application requires hardware support on both the host and device sides, modifying the settings of the link training state machine. Due to changes in the application's signal, and the need to modify circuits and components on the PCB board to optimize signal quality, this allows for asymmetrical data transmission lines, more efficient use of PCB board space resources, and more flexible support for multiple device connections.

[0081] The asymmetric PCIe bus connects the CPU on the host side and the PCIe devices on the device side, such as... Figure 2 As shown, the CPU and PCIe devices are connected via several differential signals for data transmission at the transmitting end (TX) and the receiving end (RX). MISC represents other control signals required by the PCIe protocol. In a typical PCIe x4 bus, one TX and one RX form a channel, with a total of 4 pairs of TX differential lines and 4 pairs of RX differential lines. However, in the asymmetric PCIe bus design of this embodiment, the number of TX and RX lines can be combined as needed. The number of TX lines at the transmitting end and the number of RX lines at the receiving end are different in the figure, with 6 pairs of TX signal lines and 2 pairs of RX differential lines designed to provide more downlink bandwidth within the limited PCB trace space.

[0082] In the PCIe protocol, the detection and negotiation of physical links between devices is controlled by the LTSSM state machine. In the Detect state of the LTSSM state machine, the host checks each PCIe channel. If no data is detected corresponding to the RX signal, the channel is abandoned. Therefore, in asymmetric PCIe designs, this does not comply with the PCIe protocol and requires hardware modifications to the state machine. Figure 2 In this configuration, TX_1 and TX_2 are considered auxiliary channels to TX_0. In asymmetric bus design, the specific circuit design must first be determined based on the type of PCIe device being connected. For example, if a device has one RX channel and three TX channels, then the host and motherboard must be designed with the corresponding three TX channels and one RX channel. Since the TX and RX channels are physically different, their channel numbers are fixed after the device is manufactured. For example, pins A0_DN and A0_DP will be identified as TX_0 by the firmware, and the corresponding identification information will be stored in the firmware according to the design. When the device starts negotiation, the LTSSM state machine on the device side will return the negotiation information received by TX_1 and TX_2 through RX_0. TX_0, TX_1, TX_2, and RX_0 are collectively considered as one channel.

[0083] Meanwhile, at the physical layer, in the standard PCIe protocol, data packets received from the data link layer are split according to the number of lanes and sent to the receiving end separately. In asymmetric design, the Byte Stripping component needs to be optimized to split the data packets for any number of TX signals. When the logic sublayer receives the data packet from the data link layer, it will split the data packet to the TX signal of each lane through the Byte Stripping component. The main function of Byte Stripping is to adjust the drift caused by data transmission on different lanes, which leads to data asynchrony. In the design of asymmetric bus, multiple TX signals are regarded as the same lane. It should be noted that Byte Stripping is only a logical module. Its specific circuit is directly implemented by gate circuits and is often integrated into the chip. Modifications are made by modifying its truth table and state machine and then synthesizing the gate circuits by EDA tools.

[0084] It should be noted that the state detection of the state machine includes detecting the static state and detecting the active state. This method also includes:

[0085] After the high-speed serial extended bus device on the device side enters the detection active state, the transmitting end sends a detection sequence with the transmitting end number to detect whether the receiving end is normal.

[0086] If all the detection sequences with transmitter numbers sent by all transmitters of all channels are received normally and all transmitters are identified by their numbers, then the receiver detection is normal and the high-speed serial extended bus device enters the link training state.

[0087] If all the detection sequences with transmitter numbers sent by all transmitters on all channels are not received normally, the receiver detects an anomaly, and the high-speed serial extended bus device returns to the detection standby state.

[0088] If some channels successfully receive the detection sequence with the sender's number and identify the sender by the number, after a set time interval, the senders of all channels that did not receive the detection sequence successfully will resend the detection sequence with the sender's number.

[0089] If the resent test result is the same as the previous test result;

[0090] Set all undetected channels to the electrically idle state and enter the link training state;

[0091] If the resent detection result is different from the previous detection result, the detection will return to a standstill state.

[0092] When a high-speed serial expansion bus device is in a detection idle state for more than a set time threshold, or when any channel in the high-speed serial expansion bus is detected to have exited the electrical idle state, the high-speed serial expansion bus device enters the detection active state.

[0093] The solution provided by this invention requires splitting the PCIe data bus, connecting different numbers of TX and RX signals to PCIe devices, and modifying the LTSSM state machine to identify asymmetric conditions. The specific implementation method of this invention is as follows:

[0094] The logic of the Detect state in the LTSSM state machine on both the Host and Device sides is modified. When a PCIe device enters the Detect.Active state, it sends a Receiver Detection sequence via TX and checks whether the corresponding RX channel receives the sequence. In asymmetric bus design, the Receiver Detection sequence needs to be numbered because sequences sent by multiple TXs will return through the same RX channel. The number needs to indicate which TX sent the detection sequence. If all sequences sent by TXs are received normally and all TXs are identified by their numbers, the detection can be considered successful, without having to match each Lane's TX and RX one by one. It should be noted that the Detect state machine is only a logical module, and its specific circuitry is directly implemented by gate circuits, which are often integrated into the chip. Modifications are made to its truth table and state machine, and then EDA tools are used to synthesize and generate gate circuits.

[0095] In other words, this application needs to be able to identify the state of the asymmetric bus and modify the logic of the PCIe interface physical layer so that it can send and receive data normally in the state of the asymmetric bus.

[0096] The PCIe lines on the motherboard are connected according to the actual device and business needs, and the TX and RX lines are connected from the processor to the device in an asymmetrical manner, and the corresponding peripheral lines are also changed.

[0097] The present invention provides an asymmetric PCIe bus design device. The asymmetric PCIe bus is used to connect the host end and the device end. Both the host end and the device end are provided with PCIe interface modules. The device includes a channel design module and a state machine setting module.

[0098] The channel design module is used to determine that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, it modifies the physical lines of the PCIe interface modules of the host and the device so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver.

[0099] The state machine setting module is used to modify the detection state logic of the training state machine between the host and device ends, so that information received by multiple sending ends can be returned to the host end through the receiving end in the same channel.

[0100] The state machine setting module is specifically used to modify the detection state logic of the training state machine between the host and device ends, and to return the information received by the sending end to the host end through the receiving end in the same channel.

[0101] The state machine setting module includes a sequence setting unit and a recognition unit;

[0102] The sequence setting unit is used to set the sender's number in the detection sequence sent by the sender. Detection sequences sent by multiple senders are received by the receiver of the same channel. Specifically, it is used to edit the number used to identify the sender in the start code of the detection sequence sent by the sender, so that the total length of the detection sequence remains unchanged.

[0103] The identification unit is used by the receiving end to identify the sending end by number after receiving the detection sequence.

[0104] Once a PCIe device enters active detection mode, the transmitting end sends a detection sequence with the transmitting end number to check if the receiving end is working properly. If all channels' receiving ends are working properly, the PCIe device enters link training mode. If no channel's receiving end is detected, the PCIe device returns to detection inactivity mode. If some channels correctly detect the presence of the receiving end, after a set time interval, the transmitting ends of all undetected channels resend the detection sequence with the transmitting end number. If the current detection result is the same as the previous detection result, all undetected channels are set to electrical idle state and enter link training mode. If the current detection result is different from the previous detection result, it returns to detection inactivity mode. When the PCIe device remains in detection inactivity mode for more than a set time threshold, or when it detects that any channel on the PCIe bus has exited electrical idle state, the PCIe device enters active detection mode.

[0105] This invention also provides an asymmetric PCIe bus device, including a host processor; the host processor includes logic circuitry for the following operations:

[0106] Send a command signal to a device that communicates with a host processor via a PCIe link to indicate the ratio of the transmitting end to the receiving end in the same channel; wherein the ratio is not equal to 1;

[0107] After receiving the instruction from the device that it can support the asymmetric configuration ratio, the host processor dynamically configures the channel to either sender or receiver based on the ratio of the sender to receiver, and connects the communication signal to the device through the configured channel.

[0108] In this embodiment, the host processor and the device are connected via several differential signals for data transmission at the transmitting end (TX) and the receiving end (RX). In a typical PCIe x4 bus, one TX and one RX form a channel, with a total of 4 pairs of TX differential lines and 4 pairs of RX differential lines. However, in the asymmetric PCIe bus connection of this embodiment, the number of TX and RX can be combined as needed. The number of TX at the transmitting end and RX at the receiving end are different. It is designed with 6 pairs of TX signal lines and 2 pairs of RX differential lines, that is, 1 RX and 3 TX form a channel, which can provide more downlink bandwidth in the limited PCB board trace space.

[0109] In some embodiments, the host processor includes logic circuitry for the following operations:

[0110] The mode of receiving data from a device through a single receiver; and the mode of sending data to a device through multiple transmitters.

[0111] In some embodiments, the host processor includes logic circuitry for the following operations:

[0112] The data transmission mode involves sending data to the device through a single transmitter and receiving data from the device through multiple receivers.

[0113] In some embodiments, the device further includes a PCIe interface module, which includes logic circuitry for configuring the physical layer for asymmetric transmitter and receiver configurations, and logic circuitry for processing signals of the physical layer configured for asymmetric transmitter and receiver configurations.

[0114] In some embodiments, the PCIe interface module includes logic circuitry for splitting data packets according to the ratio of senders to receivers in the same channel.

[0115] This invention also provides an asymmetric PCIe bus system, including a host processor and peripheral devices, wherein the peripheral devices include logic circuitry for the following operations:

[0116] Identify multiple devices coupled to the host processor via PCIe links;

[0117] The system identifies the read / write ratio of the host processor and the device based on the application of the device and the host processor, and provides the ratio to the host processor.

[0118] The host processor includes logic circuitry for the following operations:

[0119] Send a command signal to a device that communicates with the host processor via a PCIe link to indicate the ratio of the transmitting end to the receiving end in the same channel; wherein the ratio is not equal to 1; the ratio is the ratio provided by the peripheral device to the host processor;

[0120] After receiving the instruction from the device that it can support the asymmetric configuration ratio, the host processor dynamically configures the channel to be either a transmitter or a receiver based on the ratio of the transmitter to the receiver, and connects the communication signal to the device through the configured channel.

[0121] In this embodiment of the invention, the peripheral device may be a storage device or similar device.

[0122] In some embodiments, the host processor includes logic circuitry for the following operations:

[0123] The mode of receiving data from a device through a single receiver; and the mode of sending data to a device through multiple transmitters.

[0124] In some embodiments, the host processor includes logic circuitry for the following operations:

[0125] The data transmission mode involves sending data to the device through a single transmitter and receiving data from the device through multiple receivers.

[0126] In some embodiments, the PCIe link includes one of a PCIe x4 slot, PCIe x12 slot, PCIe x16 slot, or PCIe x32 slot that interconnects the host processor with the device.

[0127] This invention provides an asymmetric PCIe bus server, comprising a host and a PCIe device, both equipped with PCIe interface modules. The connection between the host and device is determined to have asymmetric bandwidth requirements. Based on these requirements, the physical wiring of the PCIe interface modules on both the host and device is modified, such that each channel in the host-to-device transmission channel has multiple transmitters corresponding to one receiver. The logic of the detection state of the link training state machine between the host and device is modified so that information received by multiple transmitters is returned to the host through the receiver in the same channel. The server includes a motherboard, on which both the host and PCIe device are mounted. The host includes a CPU, and the device includes an accelerator card.

[0128] The detection sequence sent by the transmitting end is set with the transmitting end's number. Detection sequences sent by multiple transmitting ends are received by the receiving end of the same channel. After receiving the detection sequence, the receiving end identifies the transmitting end by its number. In fact, the number used to identify the transmitting end is edited into the start code of the detection sequence sent by the transmitting end, so that the total length of the detection sequence remains unchanged.

[0129] The state machine's detection states include a detection idle state and a detection active state. When the high-speed serial expansion bus device enters the detection active state, the transmitting end sends a detection sequence with a transmitter number to check if the receiving end is functioning correctly. If all transmitters on all channels send detection sequences with transmitter numbers that are correctly received, and all transmitters are identified by their numbers, the receiving end is considered to have detected correctly, and the high-speed serial expansion bus device enters the link training state. If none of the transmitters on all channels send detection sequences with transmitter numbers that are correctly received, the receiving end is considered to have detected an anomaly, and the high-speed serial expansion bus device returns to the detection idle state. If some channels successfully receive detection sequences with transmitter numbers that are correctly received and the transmitters are identified by their numbers, after a set time interval, all transmitters on the channels that did not receive correctly resend the detection sequence with their transmitter numbers. If the resent detection result is the same as the previous detection result, all undetected channels are set to the electrical idle state and enter the link training state. If the resent detection result is different from the previous detection result, the device returns to the detection idle state. When a high-speed serial expansion bus device is in a detection idle state for more than a set time threshold, or when it is detected that any channel in the high-speed serial expansion bus has exited the electrical idle state, the high-speed serial expansion bus device enters the detection active state.

[0130] The asymmetric PCIe bus connects the CPU on the host side and the PCIe devices on the device side, such as... Figure 2 As shown, the CPU and PCIe devices are connected via several differential signals for data transmission at the transmitting end (TX) and the receiving end (RX). MISC represents other control signals required by the PCIe protocol. In a typical PCIe x4 bus, one TX and one RX form a channel, with a total of 4 pairs of TX differential lines and 4 pairs of RX differential lines. However, in the asymmetric PCIe bus design of this embodiment, the number of TX and RX lines can be combined as needed. The number of TX lines at the transmitting end and the number of RX lines at the receiving end are different in the figure, with 6 pairs of TX signal lines and 2 pairs of RX differential lines designed to provide more downlink bandwidth within the limited PCB trace space.

[0131] In the PCIe protocol, the detection and negotiation of physical links between devices is controlled by the LTSSM state machine, such as... Figure 3 As shown, in the Detect state of the state machine, the host detects each PCIe lane. If no data is detected corresponding to the RX signal, the lane is abandoned. Therefore, in asymmetric PCIe designs, this does not conform to the PCIe protocol and requires hardware modification of the state machine. Figure 2In this configuration, TX_1 and TX_2 are considered auxiliary channels to TX_0. In asymmetric bus design, the specific circuit design must first be determined based on the type of PCIe device being connected. For example, if a device has one RX channel and three TX channels, then the host and motherboard must be designed with the corresponding three TX channels and one RX channel. Since the TX and RX channels are physically different, their channel numbers are fixed after the device is manufactured. For example, pins A0_DN and A0_DP will be identified as TX_0 by the firmware, and the corresponding identification information will be stored in the firmware according to the design. When the device starts negotiation, the LTSSM state machine on the device side will return the negotiation information received by TX_1 and TX_2 through RX_0. TX_0, TX_1, TX_2, and RX_0 are collectively considered as one channel.

[0132] In some embodiments, the device side can also be a memory chip, which is configured not to transmit data with the host side through the PCIe bus port.

[0133] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should also be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. An asymmetric PCIe bus design method, characterized in that, An asymmetric PCIe bus is used to connect the host and the device. Both the host and the device are equipped with PCIe interface modules. The method includes the following steps: It is determined that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, the physical lines of the PCIe interface modules of the host and the device are modified so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver, or each channel is configured with multiple receivers corresponding to one transmitter. The logic of the detection state of the training state machine of the host and device links is modified so that the information received by multiple sending ends is returned to the host end through the receiving end in the same channel, or the information received by the sending end is distributed to each receiving end in the same channel and returned to the host end. The steps to modify the detection state logic of the training state machine of the host and device links include: setting the sender number in the detection sequence sent by the sender; receiving detection sequences sent by multiple senders through the receiver of the same channel; and identifying the corresponding sender by the number after receiving the detection sequence. The steps for setting the sender's number in the detection sequence sent by the sender include: Edit the number used to identify the sender in the start code of the detection sequence sent by the sender, so that the total length of the detection sequence remains unchanged.

2. The asymmetric PCIe bus design method according to claim 1, characterized in that, After modifying the logic of the detection state of the training state machine between the host and device ends, so that information received by multiple sending ends is returned to the host end through the receiving end in the same channel, the following steps are included: Perform link training on one or more channels connecting the host to the device; During link training, all configured senders in the detection channel transmit data, and all configured receivers receive data and identify the corresponding senders.

3. The asymmetric PCIe bus design method according to claim 2, characterized in that, The state machine detects states including static and active states. The steps for performing link training on one or more channels connecting the host to the device include: After the PCIe device on the device side enters the detection active state, the sending end sends a detection sequence with the sending end number to detect whether the receiving end is normal. If all the detection sequences with sender numbers sent by all senders in all channels are received normally, and all senders are identified by their numbers, then the receiver detection is normal, and the PCIe device enters the link training state. If all the detection sequences with sender numbers sent by all senders on all channels are not received normally, the receiver detects an anomaly, and the PCIe device returns to a detection standby state.

4. The asymmetric PCIe bus design method according to claim 3, characterized in that, The method also includes: If some channels successfully receive the detection sequence with the sender's number and identify the sender by the number, after a set time interval, the senders of all channels that did not receive the detection sequence successfully will resend the detection sequence with the sender's number. If the resent test result is the same as the previous test result; Set all undetected channels to electrical idle state and enter link training state; If the resent detection result is different from the previous detection result, the detection will return to a standstill state.

5. The asymmetric PCIe bus design method according to claim 4, characterized in that, The method also includes: When a PCIe device remains in a detection inactive state for more than a set time threshold, or when any channel in the PCIe bus is detected to have exited the electrical idle state, the PCIe device enters the detection active state.

6. An asymmetric PCIe bus design device, characterized in that, An asymmetric PCIe bus is used to connect the host and the device. Both the host and the device are equipped with PCIe interface modules. The device includes a channel design module and a state machine setting module. The channel design module is used to determine that the connection between the host and the device has asymmetric bandwidth requirements; and based on the asymmetric bandwidth requirements of the connection, it modifies the physical lines of the PCIe interface modules of the host and the device so that each channel in the transmission channel from the host to the device is configured with multiple transmitters corresponding to one receiver, or each channel is configured with multiple receivers corresponding to one transmitter. The state machine setting module is used to edit the number used to identify the sender in the start code of the detection sequence sent by the sender, so that the total length of the detection sequence remains unchanged, and the detection sequences sent by multiple senders are received by the receiver of the same channel; after receiving the detection sequence, the receiver identifies the corresponding sender by the number. This allows multiple sending ends to receive information and then return it to the host end through the receiving ends in the same channel, or for the sending end to receive information and then distribute it to the receiving ends in the same channel before returning it to the host end.

Citation Information

Patent Citations

  • Asymmetric lanes in point-to-point interconnect

    CN109844725A

  • Partial link width states for multi-lane links

    CN112866105A