Data protection system for PCIe switch, PCIe switch and communication method of PCIe switch

Abstract

The invention discloses a data protection system for a PCIe (Peripheral Component Interconnect Express) switch. The data protection system comprises a CRC (Cyclic Redundancy Check) module, a data verification module and an error monitoring management module, wherein the CRC verification module is in communication connection with a non-transparent bridge port, and / or an upstream port, and / or a downstream port of the PCIe switch and is used for carrying out CRC verification on data received and sent by the ports, and the data verification module is in communication connection with an internal bus of the PCIe switch and is used for verifying a data packet transmitted on the internal bus and sending the verified data packet to the PCIe switch. And the error monitoring management module is in communication connection with the verification module, and is used for monitoring the data packets sent and received by the ports to identify illegal packets, recording the number and types of the illegal packets, and feeding back according to the types of the illegal packets. The data protection system can be used for a non-transparent bridge port and the like of a PCIe (Peripheral Component Interconnect Express) switch, performs data protection on the whole transmission process of a data stream, ensures the integrity of the data, and improves the reliability.

Description

Technical Field

[0001] This invention relates to the field of PCIe technology, and in particular to a data protection system for PCIe switches, a PCIe switch, and a communication method thereof. Background Technology

[0002] Peripheral Component Interconnect Express (PCIe) is a local bus technology and interface standard. PCIe devices use independent address spaces, namely the address space of the PCIe bus, also known as a PCIe bus domain. The PCIe protocol defines three types of devices: Root Complex (RC), Switch, and Endpoint (EP). A tree topology within a PCIe bus domain contains only one RC. To connect different RCs, non-transparent bridging (NTBs) is used to isolate the address spaces of different PCIe bus domains. The PCIe switch uses high-speed PCIe interfaces to connect to external devices, expanding PCIe interfaces and interconnecting different devices with PCIe interfaces for information exchange and processing.

[0003] As a key chip in server hardware, PCIe switch chips need to meet the high reliability and data integrity requirements of the server industry. Especially in scenarios where PCIe switches have NTB functionality and connect multiple RCs, ensuring data integrity and security, and timely reporting of errors are crucial when data is transmitted through NTB. Summary of the Invention

[0004] To achieve data protection for non-transparent bridge communication and meet data integrity requirements, enabling PCIe switches to be applied in critical fields such as servers, the first aspect of this invention provides a data protection system for PCIe switches, comprising:

[0005] The CRC check module is communicatively connected to the non-transparent bridge port (NTB) and / or upstream port (USP) and / or downstream port (DSP) of the PCIe switch, and is used to perform CRC checks on the data received and sent by the port.

[0006] A data verification module, communicatively connected to the internal bus of the PCIe switch, is used to verify data packets transmitted on the internal bus; and

[0007] The error monitoring and management module, which is communicatively connected to the CRC verification module, is used to monitor the data packets sent and received by the port to identify illegal packets, record the number and type of illegal packets, and provide feedback based on the type of illegal packets.

[0008] Furthermore, the data verification module uses the ECC algorithm or the parity check algorithm to verify the data packets.

[0009] Furthermore, the error monitoring and management module includes:

[0010] The monitoring submodule is used to detect illegal data packets;

[0011] The completion packet generation submodule is used to generate a completion packet (cpl) to reply to the sending host when the illegal packet type is non-pos t, wherein the completion packet includes a flag indicating that the completion status is abnormal;

[0012] The error flagging submodule is used to flag and record the types of illegal packets;

[0013] The faulty packet storage submodule is used to store the header and at least part of the payload data of the illegal packet;

[0014] The counting submodule is used to record the number of illegal packets of each type; and

[0015] The interrupt generation submodule is used to send an interrupt notification to the system when the number of illegal packets exceeds a preset value.

[0016] Furthermore, the types of illegal packets include: packets with invalid non-transparent bridge addresses, packets with invalid non-transparent bridge IDs, packets with poisoned non-transparent bridges, packets with failed ECC verification sent or received by non-transparent bridge ports, and packets with data lengths that do not match the load.

[0017] Furthermore, the interrupt is an MSI / MSI-X interrupt.

[0018] Furthermore, the data protection system also includes:

[0019] The address translation module, which is connected between the CRC verification module and the error monitoring and management module, is used to translate the address in the data packets received or sent by the non-transparent bridge port to the specified PCIe bus domain.

[0020] Based on the data protection system described above, a second aspect of the present invention provides a PCIe switch, comprising:

[0021] At least one non-transparent bridge port that is communicatively connected to the root union of other PCIe bus domains;

[0022] The upstream port is communicatively connected to the root union of this PCIe bus domain;

[0023] At least one downstream port communicatively connected to a PCIe terminal device; and

[0024] The routing module (Crossbar) is used to send data packets to the destination port, either through the data protection system described above or by directly communicating with each port.

[0025] Based on the PCIe switch described above, a third aspect of the present invention provides a communication method thereunder, comprising:

[0026] After receiving a data packet from the root union or terminal device via the port, the data packet is verified and monitored. If the data packet passes verification and is a valid packet, it is sent to the designated port through the routing module; otherwise, an error is reported.

[0027] When a data packet is sent to a port from within the PCIe switch, an ECRC checksum is generated and appended to the data packet before it is sent to the port.

[0028] Furthermore, data packet verification includes:

[0029] The data packet is checked using a CRC check module. If an error occurs, the AER mechanism described in the PCIe SPEC is used to record the error and notify the system.

[0030] When the data packet is transmitted on the internal bus, it is verified by the data verification module. If a unit (bit) error occurs, it is corrected. If multiple bits are incorrect, they are reported to the error monitoring and management module for processing.

[0031] Furthermore, monitoring data packets includes:

[0032] The error monitoring and management module monitors data packets that have been transmitted without errors to ensure that the access address and ID of the data packets are valid and that the data length matches the load information.

[0033] Furthermore, the error message includes:

[0034] The error monitoring and management module marks error types, caches error packet information, records the number of error packets, and notifies the system of interrupted or completed packets.

[0035] This invention provides a data protection system for PCIe switches, which monitors and handles error scenarios in the received or transmitted data packets, particularly those on non-transparent bridge ports. It protects the data throughout the entire transmission process by using ECRC / ECC to ensure data integrity and by monitoring abnormal packets in real time to prevent unauthorized or abnormal access, thereby improving reliability. These measures enable the PCIe switch to be applied in critical applications such as servers. Attached Figure Description

[0036] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0037] Figure 1 This diagram illustrates the structure of a data protection system for a PCIe switch according to an embodiment of the present invention.

[0038] Figure 2 This diagram illustrates the structure of an error monitoring and management module according to an embodiment of the present invention; and

[0039] Figure 3 The diagram shows a structural schematic of a PCIe switch according to an embodiment of the present invention. Detailed Implementation

[0040] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or in conjunction with other alternatives and / or additional methods or components. In other instances, well-known structures or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific numbers and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details. Furthermore, it should be understood that the embodiments shown in the drawings are illustrative representations and are not necessarily drawn to scale.

[0041] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0042] It should be noted that the embodiments of the present invention describe the method steps in a specific order; however, this is only for illustrating the specific embodiment and not for limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to actual needs.

[0043] To address the issue of data protection in NTB communication scenarios not being addressed in existing PCIe standard protocols, this invention provides a non-transparent bridge data protection system for PCIe switches. This system can be built into the PCIe switch and connected to each port, especially the NTB port, to verify and monitor the data received or sent by each port. This ensures data integrity and improves reliability in various communication scenarios, enabling PCIe switches to meet the security requirements of critical fields such as servers.

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0045] Figure 1 This diagram illustrates a structural schematic of a data protection system for a PCIe switch according to an embodiment of the present invention. Figure 1 As shown, a non-transparent bridge data protection system for a PCIe switch is built into the PCIe switch and includes a CRC check module 101, a data check module 102, and an error detection management module 103. The CRC check module 101 is connected to the NTB port, and / or USP, and / or DSO of the PCIe switch to perform CRC check on the data received and sent by the port. The data check module 102 is used to check the data transmitted on the internal bus of the PCIe switch. The error detection management module 103 is used to monitor the data packets sent and received by the port that have passed the check, to identify illegal packets, record the number and type of illegal packets, and provide feedback based on the type of illegal packets.

[0046] In one embodiment of the present invention, the CRC verification module 101 checks the data input from external devices to the PCIe switch via the port. If the data carries an ECRC checksum, the CRC verification module 101 performs an ECRC check on the data. If an error is found, the error is recorded and reported using the AER mechanism specified in the PCIe protocol. On the other hand, for data output from the PCIe switch to the port, the CRC verification module 101 determines whether ECRC verification is enabled based on the system configuration. If enabled, the CRC verification module 101 generates an ECRC checksum and appends it to the data packet before sending it out.

[0047] In one embodiment of the present invention, the data verification module 102 uses ECC algorithm or parity check algorithm to verify the data packet. For single bit errors, it corrects them, while multi-bit errors are reported to the error monitoring and management module 103.

[0048] Figure 2 This diagram illustrates the structure of an error monitoring and management module according to an embodiment of the present invention. Figure 2 As shown, in one embodiment of the present invention, the error monitoring and management module 103 includes a monitoring submodule 131, a completion packet generation submodule 132, an error marking submodule 133, an error packet storage submodule 134, a counting submodule 135, and an interrupt generation submodule 136.

[0049] The monitoring submodule 131 is used to detect illegal data packets. In one embodiment of the present invention, the illegal data packets, referred to as illegal packets, include: data packets with invalid addresses, data packets with invalid IDs, poisoned data packets, data packets whose ECC verification failed when sent or received through the port, and data packets whose data length and load do not match, etc. For NTB scenarios, the illegal packets include packets with invalid NTB addresses, packets with invalid NTB IDs, poisoned NTB packets, packets whose ECC verification failed when passing through the NTB port, and packets whose data length and load do not match, etc.

[0050] The completion packet generation submodule 132 is used to generate a completion packet (cpl) to reply to the sending host when the illegal packet type is non-pos t, wherein the completion packet includes a flag indicating an abnormal completion status.

[0051] The error marking submodule 133 is used to mark and record the type of illegal packet. It sets a mark and records the error type of the illegal packet using the corresponding register to indicate that such an error has occurred, which is convenient for the system to read back and analyze.

[0052] The faulty packet storage submodule 134 is used to store the header and at least part of the payload data of the illegal packet.

[0053] The counting submodule 135 is used to record the number of illegal packets of each type. In one embodiment of the present invention, the counting submodule 135 includes a counter module for each error type. When an error occurs, the counter for the corresponding error type increases, facilitating software back-reading and analysis.

[0054] The interrupt generation submodule 136 is used to send an interrupt notification to the system when the number of illegal packets exceeds a preset value. In one embodiment of the present invention, when an error occurs or the error count reaches a certain level, the interrupt generation submodule 136 notifies the system by sending an MSI / MSI-X interrupt.

[0055] In one embodiment of the present invention, when the data protection system is applied to the NTB port, it further includes an address translation module 104, which is connected between the CRC verification module 101 and the error monitoring and management module 103. The address translation module 104 can translate the addresses in read / write data packets from devices such as RCs in the local PCIe bus domain to other PCIe bus domains, and conversely, it can translate the addresses in read / write data packets from other PCIe bus domains to the local PCIe bus domain, thereby enabling communication between RCs in different PCIe bus domains or between RCs and other PCIe switches.

[0056] Based on the data protection system described above, Figure 3 This diagram illustrates the structure of a PCIe switch according to an embodiment of the present invention, as shown below. Figure 3 As shown, a PCIe switch includes several ports, a data protection system 302 as described above, and a routing module 303. The ports include a USP 311, at least one NTB port 312, and at least one DSP 313. The data protection system 302 is connected to each port, particularly the NTB port 312. The routing module 303 is communicatively connected to each port via the non-transparent bridge data protection system 302 or directly to the ports to send legitimate data packets to their destination ports.

[0057] Based on the data protection system described above, the communication method of the PCIe switch includes data protection measures throughout the entire data flow transmission path. Specifically, when data enters the PCIe switch from the outside via a port, especially the NTB port, it is first verified and monitored. Only if the data packet passes verification and is a valid packet can it be sent to the designated port through the routing module; otherwise, an error is reported. When data packets inside the PCIe switch are sent to a port, especially the NTB port, if the system is configured to enable CRC, an ECRC checksum is first generated and appended to the end of the data packet before it is sent to the corresponding port.

[0058] In one embodiment of the present invention, the verification and monitoring of data packets includes:

[0059] If the system configuration supports ECRC check, the data packet is first checked by the CRC check module to ensure data integrity. If an error occurs, the AER mechanism described in the PCIE SPEC is used to record and notify the system to complete the error handling.

[0060] When data packets are transmitted on the internal bus, they are verified by a data verification module. This verification can be, for example, ECC checking or parity checking. When a single-bit error occurs, it is corrected directly. When a multi-bit error occurs, it is reported to the error monitoring and management module for processing. ECC and parity checking algorithms, as mature industrial verification schemes, can ensure data integrity during transmission within the bus.

[0061] To prevent unauthorized or abnormal access from causing system risks and to improve reliability, the error monitoring and management module will monitor data packets in real time to ensure that the access address and ID of the data packets are valid and that the data length matches the load information, thereby ensuring that the data packets are legitimate.

[0062] In one embodiment of the present invention, when any of the above-mentioned errors occurs, the error monitoring and management module will perform processing, including:

[0063] Mark error types, cache error packet information, and record the number of error packets to facilitate subsequent system readback of this information for tracking and analysis of error causes; and

[0064] The system notifies the user that for non-post packets, a completion packet marked as having an abnormal completion status is sent in response, while for post packets, the system is notified by sending an MSI / MSI-X packet.

[0065] This invention provides a data protection system for PCIe switches, which monitors and handles error scenarios in the received or transmitted data packets, particularly those on non-transparent bridge ports. It protects the data throughout the entire transmission process by using ECRC / ECC to ensure data integrity and by monitoring abnormal packets in real time to prevent unauthorized or abnormal access, thereby improving reliability. These measures enable the PCIe switch to be applied in critical applications such as servers.

[0066] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.

Claims

1. A data protection system for PCIe switches, characterized in that, The system comprises: a CRC check module, which is communicatively connected with a non-transparent bridge port, and / or an upstream port, and / or a downstream port of a PCIe switch, and is configured to perform CRC check on data received and sent by the port; a data check module, which is communicatively connected with an internal bus of the PCIe switch, and is configured to perform check on data packets transmitted on the internal bus; an error monitoring management module, which is communicatively connected with the CRC check module, and is configured to monitor data packets sent and received by the port to identify illegal packets, and record the number and type of illegal packets, and provide feedback according to the type of illegal packets. The data check module uses ECC algorithm or parity algorithm to check data packets.

2. The data protection system of claim 1, wherein, The error monitoring management module comprises:

3. The data protection system of claim 1, wherein, a monitoring sub-module, which is configured to detect illegal data packets; a completion packet generation sub-module, which is configured to generate a completion packet to reply to a packet-sending host when the type of illegal packet is non-post, wherein the completion packet comprises a mark of completion status exception; an error marking sub-module, which is configured to mark and record the type of illegal packet; an illegal packet storage sub-module, which is configured to store the packet header and at least part of the payload data of the illegal packet; a counting sub-module, which is configured to record the number of illegal packets of each type; and an interrupt generation sub-module, which is configured to send an MSI / MSI-X interrupt to notify the system when the number of illegal packets exceeds a preset value. The type of illegal packet comprises: address illegal data packet, ID illegal data packet, poisoned data packet, ECC check failed data packet sent or received by the port, and data length and payload mismatched data packet.

4. The data protection system of claim 3, wherein, Further comprising:

5. The data protection system of claim 1, wherein, an address conversion module, which is connected between the CRC check module and the error monitoring management module, and is configured to convert the address in the data packet received or sent by the non-transparent bridge port to a specified PCIe bus domain. The system comprises:

6. A PCIe switch, characterized by at least one non-transparent bridge port, which is communicatively connected with a root complex of another PCIe bus domain; an upstream port, which is communicatively connected with a root complex of the PCIe bus domain; at least one downstream port, which is communicatively connected with a PCIe terminal device; and a routing module, which is communicatively connected with each port through the data protection system as claimed in any one of claims 1 to 5, or directly, and is configured to send data packets to a destination port. The system comprises: after receiving a data packet sent by a root complex or a terminal device through a port, performing check and monitoring on the data packet, if the data packet passes the check and is a legal packet, sending the data packet to a specified port through a routing module, otherwise, reporting an error; 7. A communication method for the PCIe switch of claim 6, characterized in that, and when there is a data packet sent to a port inside the PCIe switch, generating an ECRC check code and appending it to the back of the data packet, and then sending the data packet to the port. The check on the data packet comprises: performing CRC check on the data packet by a CRC check module, and if there is an error, using AER mechanism described in PCIe SPEC to record and notify the system; and 8. The communication method of claim 7, wherein, ​ ​ The data packet is checked by a data checking module when transmitted on the internal bus, and for a unit error, correction is performed, and for a multi-bit error, reporting to an error monitoring management module for processing.

9. The communication method of claim 7, wherein, The monitoring of the data packet includes: The error monitoring management module monitors the data packet without error to ensure that the access address and ID of the data packet are legal, and the data length and load information are matched.

10. The communication method of claim 7, wherein, The error reporting includes: The error monitoring management module marks the error type, caches the error packet information, records the error packet quantity, and notifies the system through interruption or completion of the packet.

Images