A data flow control method and apparatus

Abstract

The present application belongs to the field of PCIe chip, and relates to a data flow control method and device, which utilizes the method of adjusting the sending time interval of credit update message to transfer the current congestion state information from the data receiver to the data sender between two mutually connected PCIe devices; when congestion occurs, the data receiver reduces the sending time interval of credit update message, and the data sender learns that the data receiver may be congested and performs flow control by calculating the time interval of the credit update message received twice to adjust the sending rate; if the data sender receives the credit update message with normal credit update message time interval, the normal data sending rate is restored. The present application utilizes the method of adjusting the sending time interval of credit update message to transfer the congestion state information between two PCIe devices, and adjusts the data sending rate through the obtained congestion state information, thereby avoiding causing congestion.

Description

Technical Field

[0001] This invention relates to a data flow control method and apparatus, belonging to the field of PCIe chip technology. Background Technology

[0002] Peripheral high-speed interconnect interfaces are a widely used type of interface. Two PCIe devices / appliances with PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) interfaces can be interconnected through the PCIe interface.

[0003] Generally, PCIe uses a credit buffer-based approach for flow control. The main methods of PCIe data communication include two types: non-post (NP) and post (P) reporting. For example... Figure 1 As shown, some PCIe devices have NP buffers for caching NP packets and P buffers for caching P packets. Taking the P buffer as an example, when PCIe device A and PCIe device B are communicating, PCIe device A periodically sends notification information to PCIe device B based on the size and current cache status of the P buffer. This notification informs PCIe device B of the number of P packets it can send to PCIe device A next, thereby regulating the data flow of P packets that PCIe device B can send.

[0004] On the one hand, when PCIe device A sends a large volume of any type of message to PCIe device B, it may affect PCIe device B's processing of other message types. For example, because some PCIe message transmissions have order-keeping requirements, when there are many messages in the P buffer, according to the order-keeping requirements, in some scenarios, PCIe device B must process the P messages before it can process the NP messages. In this case, the delay in NP message processing leads to an increased processing latency for NP messages on PCIe device B.

[0005] Therefore, by controlling the link traffic between PCIe devices, it is possible to effectively avoid the impact of congestion in the link's credit buffer on the packet processing performance of other link buffers.

[0006] The relevant existing technologies are as follows:

[0007] Traffic control is performed by adjusting the maximum credit value of the link credit buffer of the upstream PCIe device.

[0008] Before a data packet can be sent, the flow control mechanism must ensure that the receiving port has sufficient available buffer space to receive the packet. The data receiver updates the credit value of the upstream PCIe device by periodically sending DLLP (Data Link Layer Packet) credit update messages. The PCIe device compares the number of currently available credit values ​​at the peer end to determine whether to send data downstream. Therefore, before sending data, the PCIe device needs to obtain the maximum credit value of the link to determine the maximum size of the downstream buffer. The calculation formula is as follows:

[0009] D = (t + T) × I / L;

[0010] Where D is the maximum credit value of the credit buffer, t is the transmission link latency, T is the update period of available credit values, I is the bandwidth of the transmission link, and L is the typical message length of the transmission link. The size of D ensures that the receiver's buffer can store the continuously sent data from upstream within a time interval t+T. When sending data, the current consumed credit value (the amount of data sent), the Credits Consumed counter CC, is added to the amount of data to be sent, and compared with the current credit limit counter CL to determine whether there is sufficient space downstream to receive data packets. When the maximum credit value D of the credit buffer decreases, the CL value will decrease, thereby reducing the amount of data that the current PCIe device can send to downstream nodes and reducing bandwidth usage frequency.

[0011] To this end, PCIe devices can adjust the maximum credit value D configured in the corresponding credit buffer for each link, and adjust the credit limit value CL of the transmitting device through the maximum credit value.

[0012] The congestion information feedback in existing technologies is at a fixed period, which can easily lead to untimely feedback after congestion occurs. It is not possible to quickly transmit the current congestion status information, and congestion processing can only begin after congestion has occurred.

[0013] Based on this, the present invention is proposed. Summary of the Invention

[0014] This invention provides a data flow control method and apparatus that utilizes the adjustment of the credit update message transmission interval to transmit congestion status information between two PCIe devices. The acquired congestion status information is used to adjust the data transmission rate, thereby preventing congestion.

[0015] 1) Obtain the current congestion status through congestion monitoring. If congestion is predicted to occur, the credit update message can be sent in advance to notify the data sender without waiting for the credit update message sending cycle to arrive.

[0016] 2) When the data sender receives a credit update message, it compares the time interval between the previous credit update message reception and the normal credit update message time interval:

[0017] 3) If the time interval is the normal credit update message interval, it means that the data receiver is not congested and is sending data at the normal data transmission rate.

[0018] 4) If the time interval is less than the normal credit update message time interval, it indicates that the data receiver may be experiencing congestion, so the current data transmission rate should be reduced to send data.

[0019] The specific technical solution is as follows:

[0020] In one aspect, a data flow control method utilizes the adjustment of the credit update message transmission interval. Between two interconnected PCIe devices, the data receiver transmits current congestion status information to the data sender. When congestion occurs, the data receiver reduces the credit update transmission interval. The data sender calculates the time interval T0 between two consecutive credit update messages received to determine if the data receiver is experiencing congestion and performs flow control to adjust the transmission rate. If the data sender receives a credit update message with a normal credit update message interval of T1, it restores the normal data transmission rate.

[0021] Further improvements include the following steps:

[0022] Step 1: The data receiver sends a credit update message to the data sender, while the data sender sends a data packet to the data receiver.

[0023] Step 2: If the data receiver does not detect any potential congestion within the T0 time interval after the last credit update message is sent, it will send a credit update message to the data sender at the normal time interval at time T1.

[0024] Step 3: After receiving the credit update message, the data sender performs an inspection;

[0025] If the update message is sent at the normal credit update message interval T1, the sending rate will be restored to the normal sending state.

[0026] If the T0 time interval is lower than the normal credit update message time interval T1, the sending rate will be reduced after receiving the update message.

[0027] Step 4: The data sender waits to receive the next credit update message.

[0028] A further improvement is made to the credit update message sending processing steps, which include:

[0029] PCIe nodes maintain a timer for each channel of each link. When data is transmitted through the PCIe link, the data receiver, as a downstream node, sends DLLP credit update messages to the data sender, while the data sender, as an upstream node, sends data packets to the data receiver. The downstream node periodically sends DLLP credit update messages to the upstream node, sending the channel's storage space counter size to the upstream node. After receiving the message, the upstream node updates its local credit limit. Congestion status is monitored synchronously during each transmission cycle. If congestion occurs, a DLLP credit update message is sent to the upstream node in advance.

[0030] A further improvement is made to the congestion monitoring method, which includes:

[0031] Method 1: Use the length of the monitoring queue to predict whether congestion will occur;

[0032] Method 2: Predict whether congestion is currently occurring by monitoring the trend of queue length changes over a certain period of time.

[0033] A further improvement is made to the step of receiving credit update messages, which includes:

[0034] After receiving a DLLP credit update message from a data receiver, the PCIe device node, acting as the data sender, calculates the interval between the two messages based on the time the credit update message was received, and compares it with T0:

[0035] If it is a normal time interval, then send data at the / restore normal rate;

[0036] If the time interval is shorter than normal, it indicates that the data sender may be experiencing congestion, thus requiring a reduction in the data transmission rate.

[0037] Secondly, the data flow control device implements the aforementioned data flow control method during operation.

[0038] In this invention, the PCIe device can adjust the data transmission rate according to the congestion status of the downstream corresponding link, so as to avoid the performance impact on other message processing caused by downstream congestion.

[0039] This invention primarily relates to a flow control mechanism for data transmission. In data communication employing credit flow control, to avoid the impact of congestion on one port on the processing performance of other packets, this invention uses an adjustment to the sending period of the updated credit value (DLLP) message to transmit the congestion status of the link corresponding to the data sender (upstream node). This allows the data receiver to adjust its sending rate accordingly based on whether congestion is present.

[0040] This invention, based on existing credit value transfer methods, adjusts the flow control of PCIe devices by combining the period of sending credit value update DLLP messages with the current congestion state.

[0041] The beneficial effects of this invention are:

[0042] This invention relates to a method and apparatus for flow control during data communication between PCIe devices. Its advantage lies in that, when the data receiver detects potential congestion, it notifies the data sender to control the data transmission rate, thereby preventing congestion caused by the data receiver continuing to receive data packets at high speed. This invention utilizes existing credit update messages. When potential congestion is detected, a credit update message is sent to the data receiver in advance. The data receiver can obtain its congestion status by comparing the time interval of the received credit messages with the normal credit update message time interval, and then perform flow control accordingly. Attached Figure Description

[0043] Figure 1 This is a schematic diagram illustrating the communication method between NP and P used in the background technology;

[0044] Figure 2 This is a schematic diagram of the overall flow control process of the present invention;

[0045] Figure 3 For credit update message sending processing flow;

[0046] Figure 4 This is a schematic diagram of a queue congestion detection method;

[0047] Figure 5 Flowchart for receiving credit update messages;

[0048] Figure 6 This is an example of the flow control process of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] Example 1

[0051] This invention utilizes the adjustment of the credit update message transmission interval to transmit current congestion status information between two interconnected PCIe devices, from the data receiver (also known as the downstream node) to the data sender (also known as the upstream node). When congestion occurs, the data receiver reduces the credit update transmission interval. The data sender calculates the time interval T0 between two consecutive received credit update messages to determine if congestion is likely at the data receiver and performs flow control to adjust the transmission rate. If the data sender receives a credit update message with a normal time interval T1, it restores the normal data transmission rate. The overall flow control process is illustrated below. Figure 2 As shown, PCIe device A is the data receiver, and PCIe device B is the data sender. The specific processing flow is as follows:

[0052] Step 1: Data receiver PCIe device A sends a credit update message to data sender PCIe device B, while data sender PCIe device B sends a data packet to data receiver PCIe device A.

[0053] Step 2: After PCIe device A sends the last credit update message, if no potential congestion is detected within the T0 time interval, then at time T1, it sends a credit update message to PCIe device B at the normal time interval.

[0054] Step 3: After receiving the credit update message, the data sender PCIe device B performs an inspection;

[0055] If the update message is sent at the normal credit update message interval T1, the sending rate will be restored to the normal sending state.

[0056] If the T0 time interval is lower than the normal credit update message time interval T1, the sending rate will be reduced after receiving the update message.

[0057] Step 4: PCIe device B waits to receive the next credit update message.

[0058] The advantage of using the credit update message sending interval for flow control is that it does not require changes to the existing credit processing mechanism. By controlling the frequency and time interval of credit value sending, it is possible to effectively transmit congestion status information between upstream and downstream nodes / devices.

[0059] Example 2

[0060] Credit update message sending and processing:

[0061] like Figure 3As shown, a PCIe node maintains a timer for each channel of each link. When data is transmitted through the PCIe link, the data receiver (downstream node) sends a DLLP credit update message to the data sender (upstream node), while the data sender (upstream node) sends data packets to the data receiver (downstream node). The downstream node periodically sends a DLLP credit update message to the upstream node, informing it of the channel's storage space counter size. Upon receiving this message, the upstream node updates its local credit limit. Congestion status is monitored synchronously during each transmission cycle. If congestion occurs, a DLLP credit update message is sent to the upstream node in advance. Therefore, proper congestion monitoring configuration is crucial.

[0062] like Figure 4 One method in this study, a general congestion detection approach, is to monitor the length of the queue to predict whether congestion has occurred.

[0063] Another method, such as Figure 4 Method two involves monitoring the trend of queue length changes over a certain period of time to predict whether congestion is currently occurring.

[0064] Two combined approaches can be used: one is to consider the length of the queue when setting a congestion threshold R, and the other is to consider the possibility of sudden congestion in the queue within a short period of time, in order to better predict the current congestion status.

[0065] exist Figure 4 In this context, t0 represents the current time, and t1 represents the time after a certain period of time.

[0066] r0 represents the current length of the cache queue, and r1 represents the length of the cache queue after a certain period of time;

[0067] r T The threshold value represents the growth limit of the cache. If the size of the cache exceeds the threshold value after a period of time, it indicates that congestion may occur.

[0068] T represents the normally set time interval.

[0069] Example 3

[0070] Credit update message processing:

[0071] like Figure 5 As shown, after the PCIe device node, acting as the data sender (upstream node), receives the DLLP credit update message from the data receiver (downstream node), it calculates the interval between the two credit update messages based on the time the credit update message was received, and compares it with T0:

[0072] If it is a normal time interval, then send data at the / restore normal rate;

[0073] If the time interval is shorter than normal, it indicates that there may be congestion at the data sender (downstream node), thus reducing the data transmission rate.

[0074] Example 4

[0075] like Figure 6 As shown, the time interval between node B receiving the DLLP credit update message at time t1 and the time t0 at the previous time is t, where t = t1 - t0. Figure 6 Nodes A and B in the diagram both belong to PCIe device nodes.

[0076] If the time interval t between two consecutive data transmissions is equal to T0, the data transmission rate will be restored to / maintained at the normal data transmission rate through the rate control circuit.

[0077] If the time interval t between two consecutive data transmissions is less than T0, it indicates that the data receiver may be experiencing congestion. Therefore, the current data transmission rate is reduced through the rate control circuit.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data flow control method, characterized in that: By adjusting the credit update message transmission interval, the data receiver transmits current congestion status information between two interconnected PCIe devices. When congestion occurs, the data receiver reduces the credit update transmission interval. The data sender calculates the time interval T0 between two consecutive credit update messages received to determine if the data receiver is experiencing congestion and performs flow control to adjust the transmission rate. If the data sender receives a credit update message with a normal credit update message interval of T1, it restores the normal data transmission rate.

2. The data flow control method according to claim 1, characterized in that, Includes the following steps: Step 1: The data receiver sends a credit update message to the data sender, while the data sender sends a data packet to the data receiver. Step 2: If the data receiver does not detect any potential congestion within the T0 time interval after the last credit update message is sent, it will send a credit update message to the data sender at the normal time interval at time T1. Step 3: After receiving the credit update message, the data sender performs an inspection; If the update message is sent at the normal credit update message interval T1, the sending rate will be restored to the normal sending state. If the T0 time interval is lower than the normal credit update message time interval T1, the sending rate will be reduced after receiving the update message. Step 4: The data sender waits to receive the next credit update message.

3. The data flow control method according to claim 2, characterized in that, The credit update message transmission includes: PCIe nodes maintain a timer for each channel of each link. When data is transmitted through the PCIe link, the data receiver, as a downstream node, sends DLLP credit update messages to the data sender, while the data sender, as an upstream node, sends data packets to the data receiver. The downstream node periodically sends DLLP credit update messages to the upstream node, sending the channel's storage space counter size to the upstream node. After receiving the message, the upstream node updates its local credit limit. Congestion status is monitored synchronously during each transmission cycle. If congestion occurs, a DLLP credit update message is sent to the upstream node in advance.

4. The data flow control method according to claim 3, characterized in that, The method for monitoring the congestion state includes: Method 1: Use the length of the monitoring queue to predict whether congestion will occur; Method 2: Predict whether congestion is currently occurring by monitoring the trend of queue length changes over a certain period of time.

5. A data flow control method according to claim 2, characterized in that, Step 3 includes: After receiving a DLLP credit update message from a data receiver, the PCIe device node, acting as the data sender, calculates the interval between the two messages based on the time the credit update message was received, and compares it with T0: If it is a normal time interval, then send data at the / restore normal rate; If the time interval is shorter than normal, it indicates that the data sender may be experiencing congestion, thus requiring a reduction in the data transmission rate.

6. A data flow control device, characterized in that, When the data flow control device is in operation, it implements a data flow control method as described in any one of claims 1-5.

Images