IB network link layer state synchronization method and device
By using custom link synchronization flow control packets and initialized flow control packets in IB networks, fast restart and synchronization of link layer states is achieved, and link layer states caused by device reset and subnet manager commands are solved. Problem, improving network recovery efficiency and stability.
Patent Information
- Application Number
- CN202411097545.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-09
AI Technical Summary
In IB networks, when the link layer state restart caused by device reset or subnet manager commands, the prior art requires a physical layer reset, resulting in a long network recovery time and may cause the problem of link layer function disorder.
By using custom link synchronization flow control packets and initialized flow control packets for handshake in IB device link layer state synchronization, fast restart and synchronization of link layer state is achieved, avoiding physical layer reset, and adapting to subnet Standard behavior of the manager.
It realizes rapid recovery of link layer state, improves the efficiency of network topology and routing information update, ensures network stability and reliability, and avoids functional disorders caused by link layer state differences.
Smart Images

Figure CN119071152A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network transmission, and in particular to an IB network link layer state synchronization method and device. Background Art
[0002] IB (InfiniBand) network is widely used in high-performance computing, big data analysis, cloud computing and other fields due to its advantages of low latency, high bandwidth and scalability. IBA architecture (InfiniBand Architecture) defines a system area network for IB protocol communication, which can be divided into multiple interconnected subnets, each of which consists of terminal nodes, switches, routers and subnet managers. Figure 1 The following is a schematic diagram of a subnet topology.
[0003] The subnet manager undertakes management tasks in the subnet, and is responsible for the maintenance of the subnet topology, the maintenance of the subnet device LID, the maintenance of the subnet communication routing information, the management of the subnet device status, etc. When a new IB device is connected to the network or the IB device is restarted, the subnet manager needs to update the subnet topology and perform link building operations on the IB device. Summary of the invention
[0004] The purpose of the present application is to provide an IB network link layer state synchronization method and device, aiming to achieve link layer state synchronization and rapid recovery of link state after restart.
[0005] According to a first aspect of the present application, a method for synchronizing an IB network link layer state is provided, comprising:
[0006] When the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, it sends a link synchronization flow control packet to the peer IB device. After receiving the link synchronization flow control packet or initialization flow control packet from the peer IB device, it sends an initialization flow control packet to the peer IB device.
[0007] When the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives a link synchronization flow control packet from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
[0008] In an optional implementation manner, after receiving the link synchronization flow control packet or the initialization flow control packet of the peer IB device, sending the initialization flow control packet to the peer IB device further includes:
[0009] After the peer IB device receives the link synchronization flow control packet from the local IB device, it returns the link layer state machine of the peer IB device to the LinkDown state for restart and enters the LinkInitialize state, and returns the link synchronization flow control packet to the local IB device at the same time;
[0010] After the local IB device receives the link synchronization flow control packet or initialization flow control packet from the peer IB device, it determines that the peer IB device is also in the LinkInitialize state, sends an initialization flow control packet to the peer IB device, and completes the LinkInitialize handshake when receiving the initialization flow control packet returned by the peer device.
[0011] In an optional implementation, after the handshake is completed in the LinkInitialize state, when the IB device receives the CPortState=Arm command from the subnet manager, the link layer state machine of the IB device is controlled to enter the LinkArm state;
[0012] In the LinkArm state, when the IB device receives CPortState=Active from the subnet manager or a message activation command, the link layer state machine of the IB device is controlled to enter the LinkActive state.
[0013] In an optional implementation manner, returning the link layer state machine of the local IB device to the LinkDown state for restarting further includes:
[0014] In the LinkArm state, when a link synchronization flow control packet or a physical layer unestablished link command is received from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart;
[0015] In the LinkActive state, when receiving the link synchronization flow control packet or initialization flow control packet of the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart;
[0016] In the LinkActDefer state, when a link synchronization flow control packet is received from the peer IB device or the physical layer unlink command times out, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
[0017] According to a second aspect of the present application, there is provided an IB network link layer state synchronization device, comprising:
[0018] The handshake unit is used to send a link synchronization flow control packet to the peer IB device when the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, and after receiving the link synchronization flow control packet or initialization flow control packet of the peer IB device, send an initialization flow control packet to the peer IB device;
[0019] The restart unit is used to return the link layer state machine of the local IB device to the LinkDown state for restarting when the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives the link synchronization flow control packet of the opposite IB device.
[0020] Compared with the related art, the technical solution of this application has the following advantages:
[0021] Through a more complete IB link layer state synchronization mechanism, link layer state synchronization is systematically achieved, standard subnet manager behavior is adapted, and it has good software compatibility. When implementing a separate hardware reset of the IB device link layer, there is no need to reset the physical layer, which is conducive to rapid recovery of the link after the link layer reset, and improves the update efficiency of the network topology and routing information; when executing the subnet manager CPortState=Down command at the IB device link layer, the state restart is directly and effectively executed, which is conducive to rapid recovery of the link state and improves network data transmission efficiency; when realizing the synchronization of the link layer state, it prevents functional confusion caused by differences in the link layer states of docking devices, and improves the stability and reliability of the network.
[0022] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures and processes indicated in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. It is obvious that the drawings described below are certain embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0024] Figure 1 It is a schematic diagram of the IB network subnet topology according to the related art.
[0025] Figure 2 It is a schematic diagram of an IB network hierarchy reference model according to related technology.
[0026] Figure 3 It is a schematic diagram of an IB network link layer state machine according to related technology.
[0027] Figure 4 It is a schematic diagram of the communication status of a two-terminal IB device according to the related art.
[0028] Figure 5 is a flowchart of an IB network link layer state synchronization method according to an exemplary embodiment of the present application.
[0029] Figure 6 It is a schematic diagram of an IB network link layer state machine according to an exemplary embodiment of the present application.
[0030] Figure 7-Figure 9 It is a schematic diagram of the communication status of a two-terminal IB device according to multiple exemplary embodiments of the present application. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0032] The reference model of a single IB device can be divided into the physical layer, link layer, network layer, transport layer and upper software layer, such as Figure 2 As shown in the figure. IB devices need to establish a link at the physical layer first and then at the link layer. After both the physical layer and the link layer have established links, data messages can be transmitted. The link layer manages the link establishment status through the link layer state machine. The link layer state machine is as follows: Figure 3 shown.
[0033] The IB link layer state machine has a total of 5 states, namely LinkDown, LinkInitialize, LinkArm, LinkActive and LinkActDefer. For the specific description of each state, please refer to the IB standard protocol. Figure 3Cport XX in the command indicates that CPortState=XX of the subnet manager. For example, Cport Down indicates that CPortState=Down. The subnet manager uses these commands to control the state jump of the link layer. PHY UP indicates that the physical layer has successfully established a link, and PHY DOWN indicates that the physical layer has not established a link. Remote initialization means that the initialization flow control packet from the other end is received in the LinkActive or LinkActDefer state, so that the link layer state machine of this end returns to the LinkDown state. Message activation means that a message with correct CRC is received in the LinkArm state. The link layer sends an initialization flow control packet in the LinkInitialize state, sends a normal flow control packet in the LinkArm and LinkActive states, and does not send a flow control packet in the LinkDown and LinkActDefer states.
[0034] When an exception occurs in the data transmission of the link layer, the link layer can be reset by hardware to the LinkDown state to re-establish the link layer, or the subnet manager can send the command CPortState=Down to the link layer to return to the initial LinkDown state. When the link layer at one end restarts from LinkDown to the LinkInitialize state, it sends an initialization flow control packet; the other end receives the initialization flow control packet in the LinkActive and LinkActDefer states, which triggers the remote initialization event, causing the state machine of the other end to return to the LinkDown state for a restart.
[0035] When resetting the link layer, it is usually necessary to reset the underlying physical layer as well. After the reset is complete, the physical layer needs to be established first, and then the link layer needs to be established. The completion of these links requires the participation of the subnet manager. In a large network system, due to the heavy workload of the subnet manager, it may take up to several hours for a single IB device to resume normal link status after being reset, which has a great impact on the transmission efficiency of data messages in the network.
[0036] On the other hand, when the subnet manager executes the CPortState=Down command on the local device, if the opposite device is in LinkArm state, the link layer state of the local device will be restarted but the opposite device will not be restarted, which will cause abnormal link layer flow control credit at both ends, and further cause link layer function disorder. Figure 4For example, if the link layer of both devices is in LinkArm state, when the CPortState=Down command is executed on the local device, the local end will return to LinkDown state, and then enter LinkInitialize and LinkArm states, but the other end device will remain in LinkArm state, resulting in an abnormal situation that one end device restarts but the other end device does not restart. Similarly, if the local end is in LinkInitialize or LinkActive state and the other end is in LinkArm state, executing the CPortState=Down command on the local device will also result in link layer abnormalities.
[0037] Based on the above analysis, the present application proposes an IB network link layer state synchronization method and device. When the IB device at one end restarts its state due to a reset or a subnet manager command, the reception and sending of a custom link synchronization flow control packet and handshake are added to realize the restart and synchronization of the state of the IB device at the other end. After the link layer enters LinkInitialize from LinkDown, it first sends and receives a custom link synchronization flow control packet, then sends and receives an initialization flow control packet, and performs a handshake check in the LinkInitialize state. The custom link synchronization flow control packet can trigger a remote initialization A event, that is, when the other end receives a custom link synchronization flow control packet in the LinkArm / LinkActive / LinkActDefer state, it returns to the LinkDown state.
[0038] See also Figure 5 The IB network link layer state synchronization method exemplarily provided by the present application includes:
[0039] Step 501: When the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, a link synchronization flow control packet is sent to the opposite IB device. After receiving the link synchronization flow control packet or initialization flow control packet from the opposite IB device, an initialization flow control packet is sent to the opposite IB device.
[0040] According to the IB protocol, the format of the link layer flow control packet (FCP) is as follows:
[0041]
[0042] The OP field protocol definition has used 4'b0000 and 4'b0001, which are the initialization flow control packet OP 1 FCP and the normal flow control packet OP 0 FCP respectively. The link synchronization flow control packet customized in this solution uses the FCP with the OP field set to 4'b1010, that is, OP A FCP.
[0043] Exemplarily, when the link layer enters the LinkInitialize state from the LinkDown state, it first sends and receives the OPAFCP, and then sends and receives the initialization flow control packet OP1FCP. Specifically, the handshake is performed as follows. When entering the LinkInitialize state, OPAFCP is first sent to the other end; after receiving the OPAFCP or OP1FCP from the other end in the LinkInitialize state, OP1FCP is sent, and the handshake completion is recorded when OP1FCP is received.
[0044] Step 502: When the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives a link synchronization flow control packet from the opposite IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
[0045] Exemplarily, when the LinkArm / LinkActive / LinkActDefer state receives OPAFCP, a remote initialization A event is triggered, and the link layer state machine is returned to the LinkDown state for restart.
[0046] The link layer state machine and state transition scheme are as follows: Figure 6 and 7 shown.
[0047] In the LinkInitialize state, first send and receive OPAFCP, then send and receive OP 1FCP initialization flow control packets and complete the LinkInitialize handshake. In the subsequent LinkArm and LinkActive states, send and receive OP 0 FCP normal flow control packets.
[0048] When the peer device is in LinkArm state, the local device can be restarted by executing CPortState = Down. This can also restart the peer device. Figure 8 When the peer device receives OPAFCP in LinkArm state, it triggers remote initialization A event, returns to LinkDown state, and then enters LinkInitialize state; when both devices are in LinkInitialize state, they perform FCP handshake, and after the handshake is completed, they can enter LinkArm state, so that both ends are restarted.
[0049] The above description is based on the case where both link layers are in LinkArm state. When the local end is in LinkInitialize or LinkActive state and the other end is in LinkArm state, the link layer abnormality problem of executing CPortState=Down command can also be solved, thus making up for the loophole of CPortState=Down command in the protocol, thereby ensuring the validity of CPortState=Down command executed by the subnet manager.
[0050] When resetting the link layer, the link layers at both ends can be restarted without resetting the physical layer.
[0051] Fig. 9 Taking the case where both ends are in LinkActive state as an example, the link layer of the local device is reset. The local device enters the LinkInitialize state after entering the LinkDown state, and sends out OP AFCP. After receiving OPAFCP in the LinkActive state, the other device triggers the remote initialization A event, returns to LinkDown state, and then enters the LinkInitialize state. When both devices are in the LinkInitialize state, the FCP handshake is performed. After the handshake is completed, the device can enter the LinkArm state, and then enter the LinkActive state, so that both ends are restarted.
[0052] The reset and restart principles for other link layer states are similar.
[0053] It can be seen that the IB network link layer state synchronization method proposed in this application has the following advantages compared with the related technologies:
[0054] Through a more complete IB link layer state synchronization mechanism, link layer state synchronization is systematically achieved, standard subnet manager behavior is adapted, and it has good software compatibility. When implementing a separate hardware reset of the IB device link layer, there is no need to reset the physical layer, which is conducive to rapid recovery of the link after the link layer reset, and improves the update efficiency of the network topology and routing information; when executing the subnet manager CPortState=Down command at the IB device link layer, the state restart is directly and effectively executed, which is conducive to rapid recovery of the link state and improves network data transmission efficiency; when realizing the synchronization of the link layer state, it prevents functional confusion caused by differences in the link layer states of docking devices, and improves the stability and reliability of the network.
[0055] Accordingly, the present application exemplarily provides an IB network link layer state synchronization device in a second aspect, including:
[0056] The handshake unit is used to send a link synchronization flow control packet to the peer IB device when the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, and after receiving the link synchronization flow control packet or initialization flow control packet of the peer IB device, send an initialization flow control packet to the peer IB device;
[0057] The restart unit is used to return the link layer state machine of the local IB device to the LinkDown state for restarting when the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives the link synchronization flow control packet of the opposite IB device.
[0058] The above-mentioned device can be implemented by the IB network link layer state synchronization method provided by the embodiment of the first aspect above. The specific implementation method can be found in the description of the embodiment of the first aspect, which will not be repeated here.
[0059] It is understood that the structures, names and parameters described in the above embodiments are only examples. Those skilled in the art can also easily think of combinations and adjustments to the structural features of the above embodiments according to the use requirements, and should not limit the concept of the present application to the specific details of the above examples.
[0060] Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that he or she may still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An IB network link layer state synchronization method, characterized in that: include: When the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, it sends a link synchronization flow control packet to the peer IB device. After receiving the link synchronization flow control packet or initialization flow control packet from the peer IB device, it sends an initialization flow control packet to the peer IB device. When the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives a link synchronization flow control packet from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
2. The IB network link layer state synchronization method according to claim 1, characterized in that: After receiving the link synchronization flow control packet or the initialization flow control packet of the peer IB device, sending the initialization flow control packet to the peer IB device further includes: After the peer IB device receives the link synchronization flow control packet from the local IB device, it returns the link layer state machine of the peer IB device to the LinkDown state for restart and enters the LinkInitialize state, and returns the link synchronization flow control packet to the local IB device at the same time; After the local IB device receives the link synchronization flow control packet or initialization flow control packet from the peer IB device, it determines that the peer IB device is also in the LinkInitialize state, sends an initialization flow control packet to the peer IB device, and completes the LinkInitialize handshake when receiving the initialization flow control packet returned by the peer device.
3. The IB network link layer state synchronization method according to claim 2, characterized in that: Further including: After the handshake is completed in the LinkInitialize state, when the IB device receives the CPortState=Arm command from the subnet manager, the link layer state machine of the IB device is controlled to enter the LinkArm state; In the LinkArm state, when the IB device receives CPortState=Active from the subnet manager or a message activation command, the link layer state machine of the IB device is controlled to enter the LinkAcfive state.
4. The IB network link layer state synchronization method according to claim 1, characterized in that: The step of returning the link layer state machine of the local IB device to the LinkDown state for restarting further includes: In the LinkArm state, when a link synchronization flow control packet or a physical layer unlink command is received from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
5. The IB network link layer state synchronization method according to claim 1, characterized in that: The step of returning the link layer state machine of the local IB device to the LinkDown state for restarting further includes: In the LinkActive state, when a link synchronization flow control packet or an initialization flow control packet is received from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
6. The IB network link layer state synchronization method according to claim 1, characterized in that: The step of returning the link layer state machine of the local IB device to the LinkDown state for restarting further includes: In the LinkActDefer state, when a link synchronization flow control packet is received from the peer IB device or the physical layer unlink command times out, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
7. An IB network link layer state synchronization device, characterized in that: include: The handshake unit is used to send a link synchronization flow control packet to the peer IB device when the link layer of the local IB device enters the LinkInitialize state from the LinkDown state, and after receiving the link synchronization flow control packet or initialization flow control packet of the peer IB device, send an initialization flow control packet to the peer IB device; The restart unit is used to return the link layer state machine of the local IB device to the LinkDown state for restarting when the link layer of the local IB device is in the LinkArm, LinkActive or LinkActDefer state and receives the link synchronization flow control packet of the opposite IB device.
8. The IB network link layer state synchronization device according to claim 7, characterized in that: The handshake unit is further used for: After the peer IB device receives the link synchronization flow control packet from the local IB device, it returns the link layer state machine of the peer IB device to the LinkDown state for restart and enters the LinkInitialize state, and returns the link synchronization flow control packet to the local IB device at the same time; After the local IB device receives the link synchronization flow control packet or initialization flow control packet from the peer IB device, it determines that the peer IB device is also in the LinkInitialize state, sends an initialization flow control packet to the peer IB device, and completes the LinkInitialize handshake when receiving the initialization flow control packet returned by the peer device.
9. The IB network link layer state synchronization device according to claim 8, characterized in that: The handshake unit is further used for: After the handshake is completed in the LinkInitialize state, when the IB device receives the CPortState=Arm command from the subnet manager, the link layer state machine of the IB device is controlled to enter the LinkArm state; In the LinkArm state, when the IB device receives CPortState=Active from the subnet manager or a message activation command, the link layer state machine of the IB device is controlled to enter the LinkActive state.
10. The IB network link layer state synchronization device according to claim 7, characterized in that: The restart unit is further used for: In the LinkArm state, when a link synchronization flow control packet or a physical layer unlink command is received from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
11. The IB network link layer state synchronization device according to claim 7, characterized in that: The restart unit is further used for: In the LinkActive state, when a link synchronization flow control packet or an initialization flow control packet is received from the peer IB device, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
12. The IB network link layer state synchronization device according to claim 7, characterized in that: The restart unit is further used for: In the LinkActDefer state, when a link synchronization flow control packet is received from the peer IB device or the physical layer unlink command times out, the link layer state machine of the local IB device is returned to the LinkDown state for restart.
Citation Information
Patent Citations
Credible synchronization method of distributed network equipment
CN101136730A
High performance interconnect physical layer
CN104487958A
Data link layer initialization method and data link layer initialization device
CN105897878A
Method and system for maintaining consistency between a flow control unit and central arbiter
GB0408780D0
The signal link state control method of electronic change
KR1019960003257A
