PCIe Switch system and PCIe switch

Abstract

According to the PCIe Switch system and the PCIe switch, a PCIe hardware IP is used as a bottom layer, and a basic PCIe interface function for physical layer and link layer management is provided; and identifying and configuring all connected devices to construct a device address routing mapping table through a Soft Switch IP mounted on the PCIe hardware IP, and determining a specific target of data transmission based on the constructed mapping table so as to route the data to a specific position of the corresponding device. According to the invention, flexible equipment enumeration can be carried out by selecting a pure hardware mode or a software and hardware combined mode according to actual requirements, so that the data can be correctly and efficiently transmitted to the target endpoint from the source equipment, and the complex problems of address resolution and routing management are solved. Compared with the prior art, the method can replace the existing PCIe Switch IP of a manufacturer, provides higher flexibility and expansibility, obtains better balance in the aspect of cost control, and is suitable for various applications needing large-scale PCIe equipment connection, virtualization and high-speed data exchange.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of PCIe switch, in particular to a PCIe Switch system and a PCIe switch. BACKGROUND

[0002] The existing PCIE Switch mainly has ASIC chip products and FPGA-based implementation schemes, both of which have advantages and disadvantages in performance and flexibility but have obvious shortcomings. As a highly integrated finished hardware, ASIC chip can provide high performance, low power consumption, stable connection and powerful management capability in commercial terminal products, with high bandwidth, strong expansion and low delay characteristics. However, its finished product attribute determines that it cannot be flexibly integrated into other systems, and function modification and customization can only rely on the existing functions of the manufacturer, lacking autonomy. Moreover, the cost is high, only suitable for large-scale mass production, the development cycle is long, and it is difficult to quickly respond to market changes. On the contrary, although FPGA has high flexibility and programmability, allowing developers to customize functions and modify and optimize at any time, it meets special needs and research and development test projects, but it is difficult to match the dedicated ASIC chip in terms of demand specifications and power consumption, and cannot meet the application scenarios with strict requirements on performance and power consumption.

[0003] No matter which implementation way, the existing PCIE Switch has obvious bottleneck in function specification. In terms of function number support, the mainstream scheme has limited support for PF (Physical Function) and SR-IOV function. For example, some FPGA implementations only support 4 or 8 PFs and at most 252 SR-IOV functions, which is far from enough for connecting a large number of devices or implementing complex virtualization functions. In addition, the MSI-X interrupt support is also imperfect. This mechanism is crucial for improving the interrupt processing efficiency and response speed of PCIe devices, but the current implementation has limitations in supporting it, which will cause delay or low efficiency problem when the device processes the interrupt, affecting the overall performance and stability of the system. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a PCIe Switch system and a PCIe switch, which solves the technical problems that the existing PCIE Switch technology has poor flexibility, high cost and long cycle of ASIC chip, poor performance and power consumption of FPGA scheme, and both have limited function specification, and the industry lacks unified standards and perfect ecology, which makes it difficult to be compatible and collaborative, and the development efficiency is low, which hinders the application range.

[0005] To achieve the above object and other related objects, the present application provides a PCIe Switch system, comprising: a PCIe hardware IP, configured to provide a basic PCIe interface function of physical layer and link layer management; a Soft Switch IP, mounted on the PCIe hardware IP, configured to identify and configure all connected device to construct a device address routing mapping table, and determine a specific target of data transmission based on the constructed device address routing mapping table, so as to route data to a specific location of a corresponding device.

[0006] In an embodiment of the present application, the Soft Switch IP is provided with one Upstream Port and a plurality of Downstream Ports; wherein each Downstream Port is connected with an EP endpoint according to the address routing mapping table.

[0007] In an embodiment of the present application, the Soft Switch IP first judges the type of the message received from the host by the Upstream Port; if it is a configuration message, it performs a device enumeration operation; if it is a memory message, it performs a data routing operation.

[0008] In an embodiment of the present application, the device enumeration operation comprises: selecting to complete the enumeration of the corresponding device and the host configuration response by a pure hardware enumeration mode or a software and hardware enumeration mode, so as to construct the device address routing mapping table; wherein the pure hardware enumeration mode comprises: the logical circuit parses the message according to the preset specification requirement to perform the enumeration of the device and the host configuration response; the software and hardware enumeration mode comprises: performing a preliminary configuration operation by the logical circuit, and transmitting the configuration message to the software by unloading the message, and dynamically adjusting and configuring by the software according to the actual specification requirement.

[0009] In an embodiment of the present application, the device address routing mapping table comprises: a DP routing table and a BAR mapping table; wherein the DP routing table comprises the mapping relationship between each address and the corresponding configured Upstream Port; the BAR mapping table comprises the mapping relationship between each address and the BAR window of the configured EP endpoint.

[0010] In an embodiment of the present application, the data routing operation comprises: querying the DP routing table according to the address information in the message, matching the corresponding Downstream Port; determining the BAR window of the target EP endpoint based on the address information in the message according to the corresponding BAR mapping table in the corresponding Downstream Port, so as to determine the target device and function; checking the BDF information in the message, so as to send data to the correct target device and function.

[0011] In an embodiment of the present application, if the packet does not match the Downstream Port or the BAR window of the target EP endpoint is not determined, a host error packet is replied through the Upstream Port.

[0012] In an embodiment of the present application, the system further comprises a software interface for dynamic host configuration, monitoring traffic, debugging the system and performing advanced function settings.

[0013] In an embodiment of the present application, a PCIe adaptation module is integrated in the EP endpoint, comprising a PCIe TGT unit for correctly forwarding the PCIe request transmitted by the host to the PCIe device or the memory, and a DMA unit for direct data transmission from the PCIe device to the memory.

[0014] To achieve the above object and other related objects, the present application provides a PCIe switch, which applies the PCIe switch system.

[0015] As described above, the present application is a PCIe switch system and a PCIe switch, which has the following beneficial effects: the present application uses a PCIe hardware IP as the bottom layer to provide a basic PCIe interface function for physical layer and link layer management; a Soft Switch IP mounted on the PCIe hardware IP is used to identify and configure all connected devices to construct a device address routing mapping table, and based on the constructed device address routing mapping table, the specific target of data transmission is determined to route the data to the specific location of the corresponding device. The present application can select a pure hardware mode or a combination of software and hardware mode for flexible device enumeration according to actual needs, which ensures that the data can be correctly and efficiently transmitted from the source device to the target endpoint, and solves the complex problem of address resolution and routing management. The present application can replace the existing PCIe Switch IP of the manufacturer, provide higher flexibility and scalability, and achieve a better balance in cost control, and is suitable for various applications requiring large-scale PCIe device connection, virtualization and high-speed data exchange. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A structure schematic diagram of the PCIe switch system in an embodiment of the present application is shown.

[0017] Figure 2 A structure schematic diagram of the PCIe switch system in an embodiment of the present application is shown.

[0018] Figure 3 A specific implementation flow schematic diagram of the PCIe switch system in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] The advantages and features of the present application will become apparent from specific examples which are given by way of illustration in the description of the embodiments and from the appended claims. Other advantages and novel features of the present application will become apparent from the following detailed description when considered in conjunction with the accompanying drawings.

[0020] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. It is also possible, however, to use other embodiments and / or make several changes without departing from the spirit and scope of the application. The above detailed description should not be considered to be limiting in any way. The scope of the embodiments of the application is to be only limited by the appended patent claims. The terms used herein are for the purpose of describing particular embodiments and are not intended to be limiting. Spatially relative terms, such as "upper", "lower", "left", "right", "below", "below", "bottom", "top", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be further understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. The devices or apparatuses shown in the figures can be turned over, and / or inverted, and / or otherwise used in orientations different than that depicted in the figures, and the spatially relative terms used herein are intended to encompass such changed or different orientations.

[0021] Throughout the specification, when it is said that a certain part is "connected" to another part, it includes not only the case of "direct connection" but also the case of "indirect connection" in which other elements are interposed therebetween. In addition, when it is said that a certain part "includes" a certain constituent element, other constituent elements are not excluded unless it is specifically stated to the contrary, and it means that other constituent elements can be further included.

[0022] The terms first, second, third, etc. that are mentioned herein are used to describe various parts, components, regions, layers and / or sections, but are not limited thereto. These terms are used only to distinguish a certain part, component, region, layer or section from another part, component, region, layer or section. Therefore, the first part, component, region, layer or section described below can be referred to as the second part, component, region, layer or section within the scope of the present application.

[0023] Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including", as used herein, specify the presence of stated features, operations, elements, components, items, and / or groups thereof, but do not preclude the presence or addition of one or more other features, operations, elements, components, items, and / or groups thereof. As used herein, the terms "or" and "and / or" are to be interpreted as inclusive, i.e., as meaning one or any combination of the items. Thus, "A, B or C" or "A, B and / or C" means any of the following: A; B; C; A and B; A and C; B and C; A, B and C. An exception to this definition will occur only when a combination of elements, functions, or operations are in some way inherently mutually exclusive.

[0024] The present application provides a PCIe Switch system, using PCIe hardware IP as the bottom layer, providing the basic PCIe interface function of physical layer and link layer management; through the Soft Switch IP mounted on the PCIe hardware IP, identifying and configuring all connected device address routing mapping table, and determining the specific target of data transmission based on the constructed device address routing mapping table, to route data to the specific location of the corresponding device. The present application can select pure hardware mode or software and hardware combination mode for flexible device enumeration according to actual needs, ensuring that data can be correctly and efficiently transmitted from the source device to the target endpoint, solving the complex problem of address resolution and routing management. The present application can replace the existing PCIe Switch IP of the manufacturer, providing higher flexibility, scalability, and better balance in cost control, suitable for various applications requiring large-scale PCIe device connection, virtualization and high-speed data exchange.

[0025] The embodiments of the present application will be described in detail below with reference to the accompanying drawings, so that those skilled in the art in the technical field to which the present application pertains can easily implement the present application. The present application can be embodied in various different forms, and is not limited to the embodiments described herein.

[0026] As Figure 1 The structure diagram of a PCIe Switch system in the embodiment of the present application is shown.

[0027] The system comprises:

[0028] PCIe hardware IP, which can be implemented using existing PCIe hardware IP (such as Xilinx or Intel hardware IP), undertakes the key task of providing basic PCIe interface functions, covering the management of the physical layer (PHY) and the link layer. This module not only can be flexibly deployed on the FPGA platform, but also can be adapted to ASIC chips, showing strong versatility and compatibility. In terms of function implementation, PCIe hardware IP provides solid foundation support for PCIe protocol. The port initialization function ensures that the PCIe interface can be correctly configured and activated when the system starts, preparing for subsequent communication; the link bandwidth management dynamically adjusts the link bandwidth according to actual needs, optimizing data transmission efficiency; the basic data transmission function guarantees the stable and reliable transmission of data between the PCIe device and the root complex (Root Complex). Through these functions, PCIe hardware IP directly communicates with PCIe devices or root complexes, building a bridge between the system and external devices. Among them, Ups-Hard-IP, as the upstream port hard-core IP provided by the FPGA manufacturer inside PCIe hardware IP, and Ups-Soft-IP, as the soft-core IP provided by the ASIC manufacturer inside Soft-Switch IP, work together to achieve efficient data transmission. When the host (HOST) sends a message, the message is transmitted from the HOST port (Root Complex) to the target device through the PCIE hardware IP, realizing the data interaction between the system and external devices.

[0029] Soft Switch IP is mounted on PCIe hardware IP, realizing the soft switching IP of PCIe switch function, with device enumeration and precise routing function, which can automatically identify all devices connected to the system, and according to the characteristics and needs of the device, it can be accurately configured, and the data can be routed to the specific location of the corresponding device. In the process of identifying and configuring devices, Soft Switch IP will construct a device address routing mapping table. This mapping table records the address information of each device and the corresponding data transmission port. Based on this mapping table, Soft Switch IP can accurately determine the specific target of data transmission, and accurately route the data to the specific location of the corresponding device, ensuring that the data can efficiently and accurately reach the destination, greatly improving the data processing efficiency and reliability of the system.

[0030] In an embodiment, the Soft Switch IP sets up one Upstream Port and multiple Downstream Ports; the Upstream Port serves as the key channel for the system to communicate with the Root Complex, and undertakes the important task of transmitting data upstream and receiving host messages. Each Downstream Port is precisely connected to a specific EP (Endpoint) according to a pre-constructed address routing mapping table. This one-to-one connection enables the system to establish stable connections with multiple external devices simultaneously, meeting diverse device access requirements and providing strong support for large-scale data acquisition, multi-device collaboration, and other application scenarios.

[0031] In an embodiment, the message is sent from the HOST RC (Root Complex) and transmitted to the Soft Switch IP via the Upstream Port. After receiving the message, the primary task of the Soft Switch IP is to accurately determine the type of the message. According to the function and purpose of the message, it is mainly divided into the following three types:

[0032] MSG message: message type message, usually used for control and management operations.

[0033] MEM message: memory type message, used for data transmission, involving memory read and write.

[0034] CFG message: configuration type message, usually used for device configuration and initialization.

[0035] According to the determination result of the message type, the Soft Switch IP will perform corresponding operations:

[0036] When it is determined to be a CFG message, the Soft Switch IP will immediately perform device enumeration operation. Device enumeration is the process of identifying and configuring newly connected devices by the system, and the Soft Switch IP will identify, configure and initialize each device connected to the Downstream Port according to the information in the CFG message, establish a device address routing mapping table, and prepare for subsequent data transmission.

[0037] If it is determined to be a MEM message, the Soft Switch IP will perform data routing operation. It will determine the target device location of the data according to the pre-constructed device address routing mapping table, and accurately forward the message to the corresponding Downstream Port, thereby realizing the memory data read and write operation between the host and the device.

[0038] For the MSG message, the Soft Switch IP will perform corresponding operations or state updates on the device according to the control and management instructions in the message.

[0039] In an embodiment, the Soft Switch IP performs a device enumeration operation, first extracting configuration information from the CFG message, which covers key content such as configuration registers, device identifiers, function numbers, etc. The configuration register information is used to set various parameters of the device, the device identifier is used to uniquely identify the device, and the function number is used to determine the specific function module of the device.

[0040] The device enumeration operation aims to complete the enumeration and host configuration response of the corresponding device, and then construct a device address routing mapping table, mainly including pure hardware enumeration mode and software and hardware enumeration mode.

[0041] In the hardware enumeration mode, the logic circuit automatically assumes all enumeration and Host configuration response tasks according to pre-set specifications and requirements. Hardware, with its built-in logic circuit and state machine, independently completes a series of enumeration work such as device identification, address allocation, and function initialization. When the configuration parameters are known and do not need to be dynamically adjusted, the hardware enumeration mode is started, and the hardware directly responds to the CFG message according to the built-in configuration register and control logic. The hardware quickly parses the message through its own logic circuit and configures the corresponding DP and EP according to the pre-set configuration parameters. The entire process does not involve software-level intervention, has fast response speed, and high efficiency.

[0042] If hardware processing is no longer applicable, for example, when the system needs to be dynamically configured according to the current running state or user requirements, the software system will intervene. In the software and hardware enumeration mode, the hardware part first transmits the configuration message to the software by unloading the message. After receiving the message, the software dynamically adjusts and responds to the enumeration configuration according to the actual specifications and requirements. Hardware is only responsible for preliminary configuration operations, such as identifying the basic type of the device and setting part of the basic parameters, while subsequent more complex configuration work that needs to be flexibly adjusted according to actual conditions is completed by software. This software and hardware enumeration mode has higher flexibility and can dynamically adjust specifications and configurations according to different requirements. This makes the enumeration process adaptable to changing hardware and protocol requirements, especially suitable for scenarios that need to be adjusted according to different application scenarios or system configurations at runtime.

[0043] After the above processing, the configuration of the relevant Downstream port and EP endpoint has been initialized. At this time, the system can accurately identify the device and respond according to the message requirements, including communication and state configuration between devices, laying a foundation for subsequent data transmission and normal operation of the system.

[0044] In an embodiment, the device address routing mapping table comprises a DP routing table and a BAR mapping table.

[0045] The DP routing table mainly records the mapping relationship between each address and its corresponding configured Upstream Port. In the topology of the PCIe system, the Upstream Port serves as an interface connecting the upper-layer device (such as the root complex) and undertakes the important task of communication with the host. The DP routing table explicitly defines the target Upstream Port corresponding to each address range, so that the PCIe switch can quickly and accurately forward the message to the correct upstream channel.

[0046] Each EP endpoint connected to each Upstream Port corresponds to a BAR (Base Address Register) mapping table, which contains the mapping relationship between each address and the configured BAR window of the EP endpoint. The EP endpoint corresponds to the terminal device in the PCIe system, and each EP endpoint usually has multiple BAR windows, each window corresponding to a different functional module or memory area inside the device. The BAR mapping table records the mapping relationship between different address ranges and the BAR window of the corresponding EP endpoint, so that the switch can accurately locate the message to the specific functional area of the target device.

[0047] In the PCIe protocol, a physical device can be divided into multiple logical devices, each of which is called a Function. Each Function has its own independent configuration space, including a set of BAR registers. Therefore, different BAR windows usually correspond to the specific functional area of different Functions. Through accurate address decoding, it can be determined that the address in the message falls within the address range defined by which BAR window. Once the BAR window is determined, the configuration information of the Function to which the window belongs can be used to further determine the specific Function of the target device.

[0048] Specifically, the Soft Switch IP first extracts the configuration information from the CFG message when receiving the CFG message from the root complex, writes the Base / Limit of the Upstream Port into ups-wins-tx|rx (upstream port-windows-Transmit|Receive) to define the address range presented by the switch to the host, and writes the global address segment required by the Downstream Port and the port control parameters into corresponding dp0-wins|cfgs (downstream port 0-windows|configuration space)…dps-wins|cfg to configure the corresponding Downstream Port; and writes the BAR window and function parameters in the EP endpoint into corresponding Ep0-wins|cfgs (endpoint 0-windows|configuration space)…Eps-wins|cfgs to configure the BAR window of the corresponding EP endpoint.

[0049] In an embodiment, the data routing operation comprises:

[0050] After receiving the MEM message, the address information is first extracted from the message header. The address information contains the address range of the target device and is the key basis for subsequent routing queries. Comparing the extracted address information with the address range in the DP routing table can quickly determine which Downstream Port the message should be forwarded through. After the message reaches the corresponding Downstream Port, the switch needs to further analyze the address information in the message. At this time, the address information has been narrowed down to the range of EP endpoints connected by the Downstream Port, but it still needs to be determined which BAR window of which EP endpoint it belongs to. According to the address information in the message, the BAR mapping table is queried, and through accurate address matching, it is determined which BAR window of which EP endpoint the message should access, and then the specific Function of the target device is determined.

[0051] Specifically, first, the Soft Switch IP extracts address information from the MEM packet header. Then, the extracted address information is compared with the address range in the DP routing table. The DP routing table establishes a mapping relationship between each address range and the corresponding Downstream Port based on the Downstream Port-related information written in dp0-wins|cfgs, dps-wins|cfgs during the previous CFG packet configuration. Through this comparison operation, it can be quickly and accurately determined which Downstream Port the packet should be forwarded through. When the packet reaches the corresponding Downstream Port, since the approximate Downstream Port has been locked with the help of the DP routing table, the address information has been limited within the EP endpoint range connected by the Downstream Port. According to the address information in the packet, the BAR mapping table is queried. The BAR mapping table records the mapping relationship between each address and the BAR window of the configured EP endpoint according to the EP endpoint BAR window and function parameter written in Ep0-wins|cfgs, Eps-wins|cfgs during the CFG packet configuration stage. Through accurate address matching, it can be determined which BAR window of which EP endpoint the packet should access, and then the specific function of the target device is determined, just like under the guidance of accurate navigation.

[0052] In the PCIe protocol, each device is assigned a unique BDF (Bus-Device-Function) identifier. The BDF information consists of three parts: bus number (Bus), device number (Device), and function number (Function). It is like a device's ID number, which can accurately identify the target device and its specific function.

[0053] Although the target device and function of the packet have been determined through the DP routing table and the BAR mapping table, in order to ensure accurate delivery of data, the BDF information in the packet also needs to be checked. This is because in a complex PCIe system, there may be multiple devices with similar address ranges or BAR window configurations, and relying solely on address information may cause data to be sent to non-target devices. The BDF information is extracted from the packet, and it is determined through the DP routing table and the BAR mapping table that the target device and function of the packet. If they match, it means that the packet should indeed be sent to the target device and function, and the switch will continue to complete the data transmission process. If they do not match, the switch will discard the packet or trigger an error handling mechanism to prevent data from being processed incorrectly.

[0054] In an embodiment, if the destination address information in the packet does not match any entry in the DP routing table, a suitable Downstream Port cannot be determined to forward the packet. Alternatively, if the address in the packet is not within any preset BAR address range, the BAR window of the target EP endpoint cannot be determined. An error handling process can be triggered immediately by the hardware, and a corresponding UR CPL packet is returned to the host according to the PCIe protocol. The UR CPL packet contains some necessary fields, such as packet type identification, error code, requester identification, etc., and the host can understand the specific reason for the error by analyzing these fields. This hardware processing method has the characteristics of fast response, and can notify the host of the problem in the shortest time. In some cases, in addition to the automatic reply of the error packet by the hardware, the software can also interact with the hardware through the configuration channel to agree to reply other types of error packets. The software can flexibly configure the error handling strategy, and then select and use the error packet according to different error types and system requirements.

[0055] In an embodiment, the system further comprises a software interface; the interface supports dynamic host configuration, can flexibly adjust the PCIe switch parameters according to actual needs, and has traffic monitoring and system debugging capabilities, can master the system running status in real time, quickly troubleshoot and solve problems; can also perform advanced function settings, such as enabling SR-IOV, MSI-X and other features, fully expanding the system performance and function boundary. In addition, it can also be used for interaction configuration with external systems, greatly enhancing the openness and compatibility of the system. The existence of the software interface makes the system configuration and management more flexible, perfectly adapting to the needs of custom control and optimization in high-end applications. Among them, the device enumeration and error handling operations described above are performed through the software interface; for the device enumeration implemented through the software interface, the software can dynamically adjust and configure the device according to actual specification requirements, realizing efficient utilization of resources. For the error handling implemented through the software interface, it can interact with the hardware through the configuration channel, and both parties agree to reply other types of error packets, effectively improving the accurate processing capability of the system for various errors, providing a solid guarantee for the stable operation of the system.

[0056] In an embodiment, as Figure 2In the EP endpoint, a PCIe adapter module is integrated, including: a PCIe TGT unit for correctly forwarding the PCIe request transmitted by the host to the PCIe device or the memory; and a DMA unit for directly transmitting data from the PCIe device to the memory. The function of this module is to realize the adaptation of the PCIe target (Target) and direct memory access (DMA) functions. Through this module, the FPGA can efficiently exchange data with external devices (such as CPU or other external buses). The PCIe TGT adaptation function is responsible for correctly forwarding the PCIe request to the target device or the memory, while the DMA adaptation function supports direct data transmission from the PCIe device to the memory, reduces the participation of the CPU, and improves the efficiency and bandwidth utilization of data transmission.

[0057] The core function of the PCIe adapter module is to realize the perfect adaptation of the PCIe target (Target) and direct memory access (DMA) functions. With this module, the FPGA can smoothly and quickly exchange data with external devices such as CPU or other external buses. Specifically, the PCIe TGT adaptation function focuses on accurately guiding the PCIe request to reach the target device or the memory, laying the foundation for accurate data flow; while the DMA adaptation function greatly reduces the participation of the CPU in the data transmission process, effectively improves the efficiency of data transmission, and fully excavates the bandwidth potential, so that the data transmission performance of the whole system is significantly optimized.

[0058] In order to better describe the PCIe Switch system, specific embodiments will be described.

[0059] Embodiment

[0060] This embodiment focuses on a PCIe Switch system and describes its execution process. The process starts from the host sending a message, and then the Soft Switch IP judges the type of the message. Different types of messages (CFG, MEM, etc.) are processed differently, including the configuration of Downstream Port and EP attribute information, message ownership judgment, and read-write operation type judgment, etc. Finally, the system correctly processes and responds to the message.

[0061] The specific steps are as shown in Figure 3 , including:

[0062] Step S1: The packet is transmitted from HOST RC via PCIE Hard IP (upstream port). The HOST sends the packet, which is transmitted from the HOST port (Root Complex) to the target device via the PCIE Hard IP (upstream port). This process involves the initialization, configuration, and packet transmission of the PCIE protocol stack.

[0063] Step S2: The Soft Switch IP determines the type of the current packet and distinguishes between MSG / MEM / CFG packets. If it is a CFG packet, step S3 is performed; if it is a MEM packet, step S5 is performed.

[0064] Step S3: The Soft Switch IP extracts the CFG packet configuration information and selects hardware processing or software-hardware combined processing to configure the Downstream Port and EP attribute information.

[0065] If it is hardware processing, the hardware directly responds to the CFG packet based on known configuration parameters. The hardware responds through built-in configuration registers and control logic, without involving software intervention.

[0066] If it is software-hardware combined processing, the hardware performs preliminary configuration, and the software system dynamically adjusts and configures the hardware response to the CFG packet based on the current system state and requirements. The software modifies or updates the configuration registers through configuration managers and other mechanisms.

[0067] Step S4: After completing the CFG packet process, the Downstream Port and EP attribute module is initialized. After the CFG packet process, the configuration of the relevant Downstream port and EP endpoint has been completed, and the system can recognize and respond to the packet, including inter-device communication and state configuration.

[0068] Step S5: Determine the Downstream Port to which the MEM packet belongs. After receiving the MEM packet, it needs to be determined to which Downstream Port the packet belongs. If the packet cannot be matched to any preset DP port, the error processing flow is entered to ensure the correctness of the packet routing and avoid packet loss or incorrect transmission.

[0069] Step S6: The message passing through the DP port is further judged to belong to the EP endpoint. After passing through the Downstream port, the message needs to be further judged to belong to which EP endpoint BAR (Base Address Register) window. If the address of the message does not belong to any preset BAR address range, an error processing flow is entered. At this time, address matching and routing are performed according to the PCIE protocol and the BAR mapping relationship of the device.

[0070] Step S7: After judging that the message belongs to the EP BAR window, the BDF information to which the message belongs is further checked. By verifying the BDF information, the system can determine whether to send the message to the correct target device and function.

[0071] Step S8: It is judged whether the message belongs to the read-write operation of the USER APP or the read-write operation of the MSIX. According to the type and identification in the message, it is judged whether the message is the read-write operation of the user application program (USER APP) or the read-write operation of the MSI-X (Message Signaled Interrupts) type. The MSI-X operation is used for interrupt delivery, and the read-write operation of the user application program is used for data transmission. According to different operation types, the system will adopt different processing methods:

[0072] USER APP read-write operation: usually involves data access and transmission, which needs to be processed through memory mapping.

[0073] MSI-X read-write operation: involves sending and processing of interrupt signals.

[0074] The application provides a PCIe switch.

[0075] The PCIe (Peripheral Component Interconnect Express) switch is a key device used for expanding and flexibly managing PCIe bus connections in a computer system. It can establish an efficient communication bridge between multiple PCIe devices, realizing high-speed data transmission and resource sharing between devices.

[0076] The PCIe switch applies the PCIe Switch system described in the above embodiment.

[0077] The specific working process includes:

[0078] The message is transmitted from the HOST RC via the PCIE HardIP (upstream port). When the host (HOST) initiates a data transmission request, the message is sent from the HOST port (Root Complex) and enters the PCIe switch via the upstream port of the PCIE HardIP. The Soft Switch IP, as the core processing unit, judges the type of the message entering the switch.

[0079] If the message is a CFG message, the Soft Switch IP extracts the configuration information therein and selects a hardware processing or a combination of hardware and software processing mode according to the system requirements to configure the Downstream Port (downstream port) and EP (Endpoint, endpoint) attribute information. After completing the CFG message processing flow, the Downstream Port and the EP attribute module are initialized. At this time, the system can accurately identify the configuration information of each port and endpoint, and respond according to the message, including communication and state configuration between devices.

[0080] If the message is a MEM message, the Downstream Port to which the message belongs is first judged. That is, it is determined to which Downstream Port the message should be routed. If the message fails to match any preset DP port, the system will enter an error processing flow to avoid message loss or error transmission and ensure the accuracy of data transmission. After passing through the Downstream port, the message needs to be further judged to which EP port the BAR (Base Address Register, base address register) window belongs. If the address of the message does not belong to any preset BAR address range, it also enters the error processing flow. The system will accurately match and route the address according to the PCIE protocol and the BAR mapping relationship of the device to ensure that the message can correctly reach the target EP port.

[0081] After judging the EP BAR window to which the message belongs, the BDF (Bus, Device, Function) information to which the message belongs is further judged. The BDF information is used to uniquely identify the device in the PCIe system, and accurate BDF judgment helps the system to accurately manage and communicate with the device.

[0082] Since the implementation principle of the PCIe Switch system has been described in the foregoing embodiments, it is not repeated here.

[0083] Compared with the prior art, the following advantages are achieved:

[0084] 1. Break through the functional limitations and support more physical functions: The existing FPGA PCIe IP specification has obvious limitations in physical function (PF) support. For example, Xilinx's product can only support 4 PFs and a maximum of 252 single-root I / O virtualization (SR-IOV) functions, and Intel's product can support 8 PFs, but the number of SR-IOV functions is also limited to 252. This limitation makes the existing technology seem inadequate when connecting and managing a large number of devices with different functions in complex systems. This solution aims to expand this limitation and provide support for 32 PFs and 64 PFs. This means that more devices with independent physical functions can be integrated and managed in the same system, greatly expanding the system's hardware access capabilities and providing a solid foundation for building large-scale and diversified device clusters.

[0085] 2. Strengthen SR-IOV expansion to meet complex virtualization needs: Although existing technologies support a certain number of SR-IOV functions, the maximum number of virtual functions (Function) is only 252. When facing the growing demand for multi-device virtualization, it gradually becomes inadequate. This solution will support a maximum of 2K virtual functions. This breakthrough enables the system to allocate independent virtual functions to more virtual machines or virtual devices, enabling more fine-grained resource allocation and management. In large-scale virtualization application scenarios such as data centers, it can significantly improve resource utilization and reduce virtualization costs, while meeting the diverse needs of complex businesses for virtualization environments, providing strong support for enterprise digital transformation.

[0086] 3. Fully implement MSI-X support to improve interrupt processing efficiency: Some current FPGA PCIe IPs have limitations in MSI-X interrupt support and cannot fully utilize its advantages. This solution fully implements the MSI-X function, enabling the system to more efficiently handle multiple interrupts. In high-performance computing, data centers, and other applications that require real-time and concurrency, full support for MSI-X can ensure that the system can respond to various device interrupt requests in a timely manner, avoiding interrupt loss or delay, thereby improving the stability and performance of the entire system and providing reliable protection for the continuous operation of critical businesses.

[0087] In summary, the PCIe Switch system and the PCIe switch of the present application use PCIe hardware IP as the bottom layer to provide the basic PCIe interface function of physical layer and link layer management; through the SoftSwitch IP mounted on the PCIe hardware IP, all connected device address routing mapping tables are identified and configured, and the specific target of data transmission is determined based on the constructed device address routing mapping table to route the data to the specific location of the corresponding device. The present application can select the pure hardware mode or the combination of software and hardware mode for flexible device enumeration according to the actual needs, ensures that the data can be correctly and efficiently transmitted from the source device to the target endpoint, solves the complex problem of address resolution and routing management. The present application can replace the existing PCIe Switch IP of the manufacturer, provide higher flexibility and scalability, and obtain a better balance in cost control, and is suitable for various applications requiring large-scale PCIe device connection, virtualization and high-speed data exchange. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has high industrial utilization value.

[0088] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A PCIe Switch system, characterized in that, include: PCIe hardware IP, used to provide basic PCIe interface functions for physical layer and data link layer management; The Soft Switch IP, mounted on the PCIe hardware IP, is used to identify and configure all connected devices to build a device address routing table, and to determine the specific target of data transmission based on the built device address routing table, so as to route the data to the specific location of the corresponding device.

2. The PCIe Switch system according to claim 1, characterized in that, The Soft Switch IP has one Upstream Port and multiple Downstream Ports; each Downstream Port is connected to an EP endpoint according to an address routing mapping table.

3. The PCIe Switch system according to claim 2, characterized in that, The Soft Switch IP first determines the type of the message received from the host on the Upstream Port; if it is a configuration message, it performs a device enumeration operation. If it is a memory-based message, then a data routing operation is performed.

4. The PCIe Switch system according to claim 3, characterized in that, The device enumeration operation includes: selecting either pure hardware enumeration mode or software / hardware enumeration mode to complete the enumeration of the corresponding device and host configuration response, so as to build a device address routing mapping table; The pure hardware enumeration mode includes: the logic circuit parses the message according to the preset specification requirements to enumerate the device and the host configuration response; the software and hardware enumeration mode includes: the logic circuit performs the initial configuration operation, and transmits the configuration message to the software by unloading the message, and the software dynamically adjusts and configures it according to the actual specification requirements.

5. The PCIe Switch system according to claim 4, wherein the device address routing mapping table comprises: DP routing table and BAR mapping table; the DP routing table includes the mapping relationship between each address and its corresponding configured upstream port; The BAR mapping table includes the mapping relationship between each address and the BAR window of the configured EP endpoint.

6. The PCIe Switch system according to claim 5, characterized in that, The data routing operation includes: The DP routing table is queried based on the address information in the message to match the corresponding Downstream Port; Within the corresponding Downstream Port, the BAR window of the target EP endpoint is determined based on the address information in the message and the corresponding BAR mapping table, so as to identify the target device and function. Check the BDF information in the message to ensure that data is sent to the correct target device and function.

7. The PCIe Switch system according to claim 6, characterized in that, If the message does not match the DownstreamPort or does not determine the BAR window of the target EP endpoint, the host will reply with an error message via the UpstreamPort.

8. The PCIe Switch system according to claim 4, characterized in that, The system also includes a software interface for dynamic host configuration, traffic monitoring, system debugging, and advanced function settings.

9. The PCIe Switch system according to claim 1, characterized in that, The EP endpoint integrates a PCIe adapter module, including: a PCIe TGT unit for correctly forwarding PCIe requests transmitted by the host to PCIe devices or memory; and a DMA unit for data transfer directly from PCIe devices to memory.

10. A PCIe switch, characterized in that, The PCIeSwitch system described in any one of claims 1 to 9 is applied.

Images