PCIE bus-based control system and method

Abstract

The application provides a PCIE bus-based control system and method. The system comprises an RC device, a plurality of EP devices and a bridge device. The bridge device is used for connecting the RC device and the EP devices through a PCIE bus. The RC device is used for system initialization. The RC device is also used for obtaining device information of the EP devices, determining a first master EP device and a non-master EP device among the EP devices according to the device information of the EP devices, and sending a first control instruction to the first master EP device. The first master EP device is used for switching itself from a service function to a master function according to the first control instruction, and executing the master function. The master function is used for managing all devices connected to the PCIE bus-based control system. The non-master EP device is used for executing a corresponding service function according to service configuration information. The application can realize that the system supports more master devices.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a control system and method based on the PCIe bus. Background Technology

[0002] PCIe (Peripheral Component Interconnect Express) bus is a high-speed serial interconnect standard used to connect devices inside a computer, such as GPUs (Graphics Processing Units), network interface cards, and memory controllers. In other words, the PCIe bus is a bus that enables the interconnection of multiple modules. Compared to traditional parallel PCI and PCI-X (PCI-eXtended, a high-performance parallel bus standard) buses, the PCIe bus offers higher bandwidth, lower latency, and better scalability, and is therefore widely used in modern computers, servers, embedded systems, and high-performance computing devices.

[0003] In some high-reliability PCIe systems, dual controllers are required to ensure system reliability. Related technologies typically employ an NTB (Non-Transparent Bridge) to connect two independent controller units within the PCIe system. These two controller units then access all endpoint (EP) devices within the PCIe system.

[0004] However, the above technology only supports two main control units, which is a relatively limited number of main control units. Summary of the Invention

[0005] This invention provides a control system and method based on the PCIe bus to address the shortcomings of existing PCIe systems that use NTB technology to support two master control units, which have a limited number of supported master control units. This invention enables the PCIe system to be initialized by an RC device and the master control device to be selected from multiple EP devices. The selected master control EP device then performs the master control functions. In this way, multiple EP devices can act as master control devices, thus supporting a larger number of master control devices and improving system reliability.

[0006] This invention provides a control system based on a PCIe bus, comprising: a root endpoint RC device, multiple endpoint EP devices, and a bridging device; The aforementioned bridging device is used to connect to RC devices and various EP devices via the PCIe bus; The aforementioned RC device is used to perform system initialization for a PCIe bus-based control system. The aforementioned RC device is also used to acquire device information of each EP device, determine the first master control EP device and non-master control EP devices among multiple EP devices based on the device information of each EP device, and send a first control command to the first master control EP device; the aforementioned first control command is used to instruct the first master control EP device to switch from performing business functions to master control functions. The aforementioned first master control EP device is used to switch itself from business function to master control function according to the first control command, and execute the master control function; the aforementioned master control function is used to manage all devices connected to the control system based on the PCIe bus; The aforementioned non-master control EP devices are used to execute corresponding business functions based on the business configuration information.

[0007] According to a PCIE bus-based control system provided by the present invention, the aforementioned RC device is further configured to detect whether there is an abnormality in the first master control EP device, and if it is determined that there is an abnormality in the first master control EP device, determine the second master control EP device from the non-master control EP devices based on the device information of the non-master control EP devices, and send a second control command to the second master control EP device; the aforementioned second control command is used to instruct the second master control EP device to perform a function switch from business function to master control function; The aforementioned RC device is also used to receive and store the current system configuration information and service configuration information transmitted by the first master control EP device; The aforementioned second master control EP device is used to switch itself from business function to master control function according to the second control command, obtain system configuration information and business configuration information from the RC device, and execute master control function according to the system configuration information and business configuration information; The aforementioned RC device is also used to, in the event that the second master control EP device is found to be abnormal, identify a third master control EP device among the non-master control devices other than the second master control EP device, designate the third master control EP device as the new second master control EP device, and return to execute the aforementioned step of sending the second control command to the second master control EP device.

[0008] According to the present invention, a control system based on a PCIE bus is provided, wherein the RC device includes a PCIE enumeration unit, an interrupt remapping unit, a configuration storage unit, a master control watchdog unit, and a master control switching unit. The aforementioned PCIE enumeration unit is used to perform system initialization for the PCIE bus-based control system. The aforementioned interrupt remapping unit is used to map the interrupt routine of the PCIe bus-based control system to the master control EP device currently executing the master control function; the aforementioned master control EP device includes any one of the first master control EP device, the second master control EP device, and the third master control EP device; The aforementioned configuration storage unit is used to receive and store the current system configuration information and service configuration information of the control system based on the PCIe bus. The aforementioned system configuration information includes the allocation of master control EP devices and non-master control EP devices and the functional status of the master control EP devices. The aforementioned service configuration information is reported by the master control EP devices to the RC devices, including the configuration information of the service functions to be executed by the non-master control EP devices. The aforementioned main control watchdog unit is used to monitor whether each EP device is online, obtain the online status of each EP device, and obtain the device information of each EP device based on the online status of each EP device. The aforementioned master control switching unit is used to determine the master control EP device and the non-master control EP device, and to send control commands to the master control EP device; the aforementioned control commands include a first control command or a second control command.

[0009] According to a PCIE bus-based control system provided by the present invention, the device information of the EP device includes the online status and device category of the EP device, wherein the device category includes a first category or a second category; The aforementioned main control switching unit is specifically used to determine candidate EP devices among the EP devices based on the online status of each EP device; the aforementioned candidate EP devices are EP devices whose online status is online. Based on the equipment category of the candidate EP devices, a first master control EP device is determined among the candidate EP devices, and the other EP devices in each EP device are determined as non-master control EP devices; the aforementioned first master control EP device is a candidate EP device whose equipment category is the first category.

[0010] According to the PCIE bus-based control system provided by the present invention, the first category is used to indicate that the EP device is an EP device using a system-on-a-chip (SOC) chip, and the second category is used to indicate that the EP device is an EP device using a non-SOC chip.

[0011] According to the PCIE bus-based control system provided by the present invention, the plurality of EP devices include at least two EP devices of the first category.

[0012] According to a PCIE bus-based control system provided by the present invention, the bridging device includes a first-level bridging device and at least one second-level bridging device, wherein the second-level bridging device is connected to the first-level bridging device.

[0013] The present invention also provides a control method based on a PCIe bus, applied to the above-mentioned PCIe bus-based control system, the method comprising: The bridging device connects to the RC device and each EP device via the PCIe bus; The RC device performs system initialization for the PCIe bus-based control system, acquires device information of each EP device, determines the first master control EP device and non-master control EP devices among multiple EP devices based on the device information of each EP device, and sends a first control command to the first master control EP device; the first control command is used to instruct the first master control EP device to switch from performing business functions to master control functions. The first master control EP device switches itself from business function to master control function according to the first control command, and executes the master control function; the aforementioned master control function is used to manage all devices connected to the PCIe bus-based control system; Non-master control EP devices execute corresponding service functions based on service configuration information.

[0014] According to a control method based on a PCIe bus provided by the present invention, the method further includes: The aforementioned RC device detects whether the first master control EP device is abnormal, and if it is determined that the first master control EP device is abnormal, it determines the second master control EP device among the non-master control EP devices based on the device information of each EP device, and sends a second control command to the second master control EP device; the aforementioned second control command is used to instruct the second master control EP device to perform a function switch from business function to master control function; The aforementioned RC device is also used to receive and store the current system configuration information and service configuration information transmitted by the first master control EP device; The aforementioned second master control EP device is used to switch itself from business function to master control function according to the second control command, obtain system configuration information and business configuration information from the RC device, and execute master control function according to the system configuration information and business configuration information; The aforementioned RC device is also used to, in the event that the second master control EP device is found to be abnormal, identify a third master control EP device among the non-master control devices other than the second master control EP device, designate the third master control EP device as the new second master control EP device, and return to execute the aforementioned step of sending the second control command to the second master control EP device.

[0015] According to a PCIe bus-based control method provided by the present invention, the device information of the EP device includes the online status and device category of the EP device, wherein the device category includes a first category or a second category; the method of determining the first master EP device and non-master EP devices among multiple EP devices based on the device information of each EP device includes: Based on the online status of each EP device, candidate EP devices are determined from among all EP devices; candidate EP devices are those whose online status is online. Based on the equipment category of the candidate EP devices, a first master control EP device is determined among the candidate EP devices, and the other EP devices in each EP device are determined as non-master control EP devices; the aforementioned first master control EP device is a candidate EP device whose equipment category is the first category.

[0016] The present invention provides a PCIe bus-based control system and method. The PCIe bus-based control system includes an RC device, multiple EP devices, and a bridging device. The bridging device connects to the RC device and each EP device via the PCIe bus. The RC device initializes the PCIe bus-based control system, acquires device information from each EP device, determines a first master EP device and non-master EP devices based on the device information, and sends a first control command to the first master EP device. The first master EP device switches itself from a business function to a master control function according to the first control command and executes the master control function. The non-master EP devices execute corresponding business functions according to business configuration information. The first control command instructs the first master EP device to switch from a business function to a master control function. The master control function manages all devices connected to the PCIe bus-based control system. This PCIe bus-based control system allows RC devices to select the master controller from among the EP devices using device information. This means any EP device can potentially become the master controller, enabling multi-master functionality. The number of master controllers is determined by the number of EP devices, thus supporting a wider range of masters. Furthermore, when the master EP device is functioning correctly, other EP devices can be configured to perform service functions using service configuration information, achieving higher performance. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a PCIE system architecture diagram that uses NTB technology to implement dual-master control devices in related technologies.

[0019] Figure 2 This is one of the architecture diagrams of a control system based on the PCIE bus provided in the embodiments of the present invention.

[0020] Figure 3 This is a structural block diagram of the RC device and EP device provided in the embodiments of the present invention.

[0021] Figure 4 This is the second architecture diagram of the control system based on the PCIE bus provided in the embodiments of the present invention.

[0022] Figure 5 This is a flowchart illustrating the control method based on the PCIe bus provided in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the specific process of the control method based on the PCIe bus provided in the embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The PCIe bus is a bus that enables the interconnection of multiple modules. With the widespread application of PCIe and the rapid development of embedded SOCs (System on Chip), many high-performance SOCs can be used as dedicated devices, such as for accelerated computing, video processing, etc. At the same time, there are also high-performance embedded processors used for business management and control.

[0026] Currently, in some systems with high reliability requirements, dual controllers are needed to ensure system reliability. Traditional PCIe modular systems generally use x86 processors as the main controller. Other technologies employ NTB technology; see [link to relevant documentation]. Figure 1 The related technology illustrated uses NTB technology to implement a PCIe system architecture with two master controllers (RC0 and RC1 in the diagram). Two master controllers (RC0 and RC1 in the diagram) establish two PCIe systems through an NTB-supporting PCIe bridge and two bridging devices (SWITCH1 and SWITCH2 in the diagram), thereby accessing all EP devices (EP0~EP12 in the diagram) and realizing system functions. However, the PCIe bridge supporting NTB is expensive and only supports a maximum of two master controllers, meaning its supported master controller units are relatively limited. Therefore, this invention provides a control system and method based on the PCIe bus, which can solve the above-mentioned technical problems.

[0027] The following examples first describe a control system based on the PCIe bus.

[0028] Figure 2This is one of the architecture diagrams of a PCIe bus-based control system provided in this embodiment of the invention. See also... Figure 2 As shown, the PCIe bus-based control system includes: a root endpoint RC device, multiple endpoint EP devices, and a bridging device. The bridging device is used to connect to the RC device and each EP device via the PCIe bus. The RC device is used to initialize the PCIe bus-based control system. The RC device is also used to acquire device information of each EP device, determine a first master EP device and non-master EP devices among the multiple EP devices based on the device information, and send a first control command to the first master EP device. The first control command is used to instruct the first master EP device to switch from a business function to a master control function. The first master EP device is used to switch itself from a business function to a master control function according to the first control command and execute the master control function. The master control function is used to manage all devices connected to the PCIe bus-based control system. The non-master EP devices are used to execute corresponding business functions according to business configuration information.

[0029] The aforementioned bridging device can be referred to as SWITCH, bridging unit, conversion device / unit, slot, etc., or simply SW. This bridging device can include one or more bridging devices. If it includes one bridging device, it is a first-level bridging device, and the EP device can be directly connected to this first-level bridging device. If it includes multiple bridging devices, the first bridging device can be referred to as SWITCH1, and subsequent bridging devices can be called second-level bridging devices. Subsequent second-level bridging devices can be directly connected to the first-level bridging devices, or they can be connected sequentially to the preceding second-level bridging device. The specific number of bridging devices connected to each bridging device can be determined based on the number of interfaces or slots on the bridging device. Each bridging device can include multiple interfaces or slots. The bridging device can be connected to multiple EP devices, bridging devices, or RC devices through multiple interfaces or slots using a PCIe bus. The connection method can be a fixed connection or a pluggable connection. It should be noted that the first-level bridging device in this embodiment can be directly connected to the RC device, while subsequent bridging devices may not be directly connected to the RC device.

[0030] The EP devices in this embodiment can include multiple devices, such as the six EP devices EP0 to EP5 shown in the figure. Of course, there can be even more EP devices. EP devices can be devices of various categories, such as graphics cards, network cards, and storage controllers, or they can be devices based on a System-on-a-Chip (SoC). In this embodiment, the device categories of each EP device can be the same (i.e., all devices of the same type), or they can all be devices of different types (i.e., all devices of different types), or some EP devices can be of the same type while others are of different types. This embodiment does not specifically limit this. It should be noted that at least two of the EP devices in this embodiment can perform master control functions. This facilitates the subsequent implementation of a system supporting at least two master control devices, enabling multi-master control functionality. Furthermore, these EP devices capable of performing master control functions can simultaneously execute master control functions and service functions, or they can execute only one of these functions at a time. They can switch between master control functions and service functions according to the control commands from the RC device. EP devices that cannot perform master control functions can only execute service functions and cannot switch functions. Different EP devices can perform different service functions. That is, there can be EP devices with different service functions, such as EP devices that can perform PCIe to network function, EP devices that can perform PCIe to image processing function, and so on.

[0031] The aforementioned RC device can be a chip on the system motherboard, such as a processor, network card, or storage controller. The aforementioned bridging device connects the RC device and each EP device via a PCIe bus, thus forming the PCIe bus-based control system of this embodiment. The RC device primarily performs system initialization for the PCIe bus-based control system of this embodiment, enabling all EP devices in the system to be accessed by the system. Simultaneously, the RC device is also used to obtain the device information of each EP device during system initialization. The device information of each EP device may include its online status, device category, device ID, service configuration information, etc. The device category reflects whether the corresponding EP device can perform the master control function. Then, under normal system functionality, based on the device information of each EP device, the EP device capable of performing the master control function can be determined and designated as the first master control EP device. The remaining EP devices can be designated as non-master control EP devices. Here, when determining the first master control EP device, it can be based on sorting the EP devices by ID from smallest to largest, finding the EP device with the smallest ID that is online and whose device category is capable of performing the master control function, and designating it as the first master control EP device. The service configuration information can specifically be the configuration information of the specific service functions that the EP device needs to perform, such as the configuration information when performing computing services, the configuration information when performing PCIe to network services, the configuration information when performing PCIe to image processing services, etc.

[0032] Each of the aforementioned EP devices executes its corresponding business function by default. When the RC device identifies the first master EP device, which may have previously been executing a business function, the RC device can send a first control command to the first master EP device to instruct it to switch its function from a business function to a master function and then execute the business function. This master function refers to the fact that the first EP master device can manage all devices connected on the PCIe bus, including the RC device, other non-master EP devices, and bridge devices. For example, the master function may include the master EP device sending business configuration information to non-master EP devices to guide them to accurately execute the corresponding business functions.

[0033] Of course, after the RC device identifies the non-master EP devices, each non-master EP device can execute its own service functions. Specifically, the first master EP device currently performing master functions can send corresponding service configuration information to the non-master EP devices. The non-master EP devices can then execute the corresponding service functions based on the acquired service configuration information. The service functions executed by different non-master EP devices may be the same or different. Furthermore, the aforementioned service configuration information can be generated by the first master EP device based on service configurations sent by external devices. During multiple master EP device switching processes, each newly identified master EP device can upload its current service configuration information to the RC device for storage. This allows the taking over master EP device to quickly obtain the service configuration information configured by the previous master EP device after a subsequent master EP device switch.

[0034] It should be noted that each EP device generally performs only one of the main control function and the business function at present. However, when the performance of the EP device is high, the EP device can perform both the main control function and the business function after being identified as the main control EP device.

[0035] Understandably, a system typically has only one master control device performing the master control function at any given time. Multi-master control function refers to the existence of other master control devices that can replace the currently faulty or abnormal master control device to perform the master control function when other master control devices fail or malfunction.

[0036] As described above, in this PCIe bus-based control system, the RC device performs system initialization and determines the function of the master EP device among the various EP devices. The master control function of the entire system is executed by the determined master EP device, eliminating the need for the RC device to handle the overall system's business control and management master control functions. This separates the system initialization function from the master control function, reducing the burden on the RC device and ensuring system reliability. Furthermore, the RC device can select the master device from multiple EP devices, and given the large number of EP devices, a large number of master devices can be selected, thus achieving the goal of supporting multiple master devices and enabling a greater number of supported master devices.

[0037] It should be noted that generally a bridging device can only connect to one RC device. Only bridging devices that support NTB functionality can connect to two RC devices. In this embodiment, the bridging device does not need to support NTB functionality; it can be a regular bridging device. This can reduce system costs while implementing multi-master functionality.

[0038] In this embodiment, the PCIe bus-based control system includes an RC device, multiple EP devices, and a bridging device. The bridging device connects to the RC device and each EP device via the PCIe bus. The RC device initializes the PCIe bus-based control system, acquires device information of each EP device, determines a first master EP device and non-master EP devices among the multiple EP devices based on the device information, and sends a first control command to the first master EP device. The first master EP device switches itself from a business function to a master function according to the first control command and executes the master function. The non-master EP devices execute corresponding business functions according to business configuration information. The first control command instructs the first master EP device to switch from a business function to a master function. The master function manages all devices connected to the PCIe bus-based control system. This PCIe bus-based control system allows RC devices to select the master controller from among the EP devices using device information. This means any EP device can potentially become the master controller, enabling multi-master functionality. The number of master controllers is determined by the number of EP devices, thus supporting a wider range of masters. Furthermore, when the master EP device is functioning correctly, other EP devices can be configured to perform service functions using service configuration information, achieving higher performance.

[0039] The following examples illustrate how initialization is performed after modularizing the RC device, and how the master control EP device and non-master control EP devices are determined.

[0040] Figure 3 This is a structural block diagram of the RC device and EP device provided in the embodiments of the present invention. See also... Figure 3 As shown, the RC device includes a PCIE enumeration unit, an interrupt remapping unit, a configuration storage unit, a master control watchdog unit, and a master control switching unit.

[0041] The aforementioned PCIE enumeration unit is used to perform system initialization on the PCIE bus-based control system, enabling all EP devices in the system to be accessed by the system. This PCIE enumeration unit is also used to obtain device information for each EP device during system initialization. This device information may include the EP device's online status, device category, device ID, and other information.

[0042] The aforementioned interrupt remapping unit is used to map the interrupt routines of the PCIe bus-based control system to the master control EP device currently performing the master control function. The master control EP device includes any one of the first master control EP device, the second master control EP device, and the third master control EP device. That is, all interrupt routines in the system are mapped to the master control EP device currently responsible for performing the master control function, so that the master control EP device can better execute the interrupt routines, better manage and control other devices, and thus realize the master control function.

[0043] The aforementioned configuration storage unit is used to receive and store system configuration information and service configuration information of the PCIe bus-based control system. The system configuration information includes the allocation of master control EP devices and non-master control EP devices, as well as the functional status of the master control EP devices. The service configuration information is reported by the master control EP devices to the RC devices, including configuration information of the service functions required to be executed by the non-master control EP devices. The configuration storage unit can store the current system configuration information and service configuration information in real time to adapt to real-time service changes. The system configuration information represents the system allocation information of the PCIe bus-based control system, specifically including the currently allocated master control EP devices and non-master control EP devices, and the functional status of the master control EP devices in the current system, i.e., whether the master control EP devices are functioning normally. By storing this system configuration information, it is convenient for each EP device to accurately and promptly obtain the overall system allocation status when switching master control EP devices among EP devices. The aforementioned service configuration information can be the specific service configuration information that the master control EP device needs to execute when managing various devices, and then send it to the configuration storage unit of the RC device for storage, so that the master control EP device determined at other times can obtain the service configuration information from the RC device in a timely manner.

[0044] The aforementioned master watchdog unit is used to monitor whether each EP device is online, obtain the online status of each EP device, and retrieve the device information of each EP device based on its online status. This master watchdog unit is mainly used by the RC device to monitor the presence of all EP devices. It can monitor the presence status by detecting whether any abnormalities or plugging / unplugging occur in each EP device, or it can monitor the presence status through hardware signals. After obtaining the presence status of each EP device, the master watchdog unit can obtain the online status of each EP device, and then combine this information with the device category, device ID, and other information obtained by the PCIE enumeration unit during system initialization to obtain the device information of each EP device.

[0045] The aforementioned master control switching unit is used to determine the master control EP device and non-master control EP devices, and to send control commands to the master control EP device; the control commands include a first control command or a second control command. This master control switching unit can, at the initial moment after system initialization, under normal conditions for all EP devices and the system, select an EP device capable of performing master control functions as the first master control EP device based on the device information of each EP device, and designate the other EP devices as non-master control EP devices; or, after the system has switched master control EP devices at least once, when the master control watchdog unit detects an anomaly in the current first or second master control EP device, the master control switching unit reselects a new master control EP device and sends a control command to it to notify it to switch functions and execute master control functions. Wherein, when the master control watchdog unit detects an anomaly in the current master control EP device, it can select a suitable EP device from the current non-master control EP devices based on their device information to become the master control EP device.

[0046] Based on the modular design of RC devices, EP devices can also be modularly designed. Optionally, refer to the above. Figure 3 Each of the aforementioned EP devices may include a watchdog feeding unit, a function switching unit, a main control function unit, and a service function unit. The watchdog feeding unit is primarily used to detect the current presence status of its own EP device and notify the RC device or the main control watchdog unit within the RC device of the current EP device's presence status. When an EP device experiences an anomaly or is plugged in / out, it cannot properly feed the watchdog, which will cause the RC device or the main control watchdog unit within the RC device to detect the anomaly. The function switching unit is primarily used to control the function modules executed / running by the current EP device. For example, an EP device acting as a main control function (e.g., EP0 in the diagram) executes the main control function, while EP devices acting as service functions (e.g., EP1~EP5 in the diagram) execute service functions. When the RC device or the main control switching unit of the RC device issues a function switching control command to the EP device or the function switching unit of the EP device, the EP device switches its function, from main control function to service function or vice versa, and executes the switched function. The main control function unit and the service function unit are the functional units of the EP device in different states. For example, when switching to the main control function, the main control function is executed through the main control function unit. When switching to the service function, the service function is executed through the service function unit.

[0047] It is understandable that the above modular design of RC and EP devices involves dividing them into multiple modules to perform different functions. The module division is only an example. In the future, different modular systems can be set up according to the different functions of the split modules, such as distributed computing systems with modular computing functions, plug-in video processing matrices with flexible expansion / change of functions, and network server systems with modular network processing, etc.

[0048] In this embodiment, by dividing the RC device and EP device into multiple modules to perform different functions, a complex system / device is broken down into multiple simple and universal modules through modular design, which makes the entire system more scalable, more flexible and more reliable.

[0049] The above embodiments illustrate the process of the RC device selecting the master EP device for the first time. If the master EP device selected in the first selection is abnormal, it may be necessary to select the master EP device again. The following embodiments illustrate this process.

[0050] In some embodiments, the RC device is further configured to detect whether the first master control EP device is abnormal, and if it is determined that the first master control EP device is abnormal, determine a second master control EP device among the non-master control EP devices based on the device information of the non-master control EP devices, and send a second control command to the second master control EP device; the second control command is used to instruct the second master control EP device to perform a function switch from a service function to a master control function; the RC device is further configured to receive and store the current system configuration information and service configuration information transmitted by the first master control EP device; the second master control EP device is configured to switch itself from a service function to a master control function according to the second control command, obtain system configuration information and service configuration information from the RC device, and perform a master control function according to the system configuration information and service configuration information; the RC device is further configured to, if it is determined that the second master control EP device is abnormal, determine a third master control EP device among the non-master control devices other than the second master control EP device, use the third master control EP device as the new second master control EP device, and return to the step of sending the second control command to the second master control EP device.

[0051] In this system, the watchdog unit in the first master control EP device continuously notifies the master control watchdog unit in the RC device of its own presence status. If the watchdog unit of the first master control EP device fails to feed the watchdog properly, causing the master control watchdog unit of the RC device to detect an anomaly, malfunction, or problem in the first master control EP device, the master control switching unit of the RC device can use the device information of the aforementioned non-master control EP devices to find the EP device with the smallest device ID that is online and of a category capable of performing master control functions. This device will be designated as the new master control EP device, recorded as the second master control EP device. The remaining non-master control EP devices will continue to function as non-master control EP devices. The master control switching unit of the RC device can send a second control command to the second master control EP device to switch it from business functions to master control functions and execute master control functions. The second master control EP device will replace the first master control EP device in continuing to execute master control functions; that is, the first master control EP device will no longer execute master control functions.

[0052] Then, the interrupt remapping unit of the RC device can remap the interrupt routines of all devices in the system to the second master EP device. At the same time, both the first and second master EP devices can obtain the service configuration information of the current non-master EP devices and send it to the configuration storage unit of the RC device for storage, so that the master EP devices determined at other times can obtain the service configuration information from the RC device.

[0053] When the aforementioned second master control EP device performs the master control function, it is also used to obtain the system configuration information and service configuration information stored in the RC device of the previous master control EP device, and then determine the current non-master control EP device based on the obtained system configuration information and service configuration information, and send the service configuration information to the current non-master control EP device to enable it to perform the service function.

[0054] Furthermore, when the aforementioned second master control EP device malfunctions, it can be removed from the current non-master control EP devices, and a new master control EP device can be identified from the remaining non-master control EP devices, designated as the third master control EP device. This third master control EP device is then used as the new second master control EP device, and the process returns to the step of the RC device sending the second control command to the second master control EP device. That is, the RC device can send the second control command to the new second master control EP device. The new second master control EP device can then switch itself from service function to master control function according to the second control command, obtain system configuration information and service configuration information from the RC device, and execute master control functions based on the system configuration information and service configuration information. Simultaneously, the new second master control EP device can also transmit the current system configuration information and service configuration information to the RC device.

[0055] If the newly determined second master control EP device malfunctions again, the steps for selecting a new second master control EP device described above can be followed to select a new master control EP device and execute the corresponding master control functions. Not all cases will be listed here.

[0056] In this embodiment, when an anomaly is detected in the current master EP device, a new EP device can be selected from the non-master EP devices as the new master EP device. This allows for flexible configuration of the master device, ensuring the reliability of the entire system. Furthermore, having more than two master devices further enhances the overall system reliability.

[0057] The following embodiments illustrate the process by which the RC device determines the first master EP device based on the device information of the EP device.

[0058] In some embodiments, the device information of the EP device includes the online status of the EP device and the device category, wherein the device category includes a first category or a second category.

[0059] The aforementioned main control switching unit is specifically used to determine candidate EP devices among the EP devices based on the online status of each EP device; the aforementioned candidate EP devices are EP devices whose online status is online. Based on the equipment category of the candidate EP devices, a first master control EP device is determined among the candidate EP devices, and the other EP devices in each EP device are determined as non-master control EP devices; the aforementioned first master control EP device is a candidate EP device whose equipment category is the first category.

[0060] The first and second categories mentioned above are different. The first category can indicate that the EP device can / is capable of performing main control functions, while the second category can indicate that the EP device cannot perform main control functions. Optionally, the first category is used to indicate that the EP device uses a System-on-Chip (SOC) chip, and the second category is used to indicate that the EP device uses a non-SOC chip. That is, the EP device of the first category is an EP device that primarily uses an SOC chip or SOC device, and this type of EP device can be responsible for complex computational decomposition, two or more high-performance processing processes such as video image processing, etc., where the SOC can be an embedded SOC; the EP device of the second category is an EP device that primarily uses a non-SOC chip or non-SOC device.

[0061] Optionally, the aforementioned plurality of EP devices may include at least two EP devices of the first category, that is, at least two EP devices that use SOC chips or SOC devices as the main components. This can realize the multi-master function of the PCIE system without the need for RC devices to perform master functions, thereby improving system performance and enhancing system reliability.

[0062] The online status of the aforementioned EP devices includes both online and offline states. When determining the first master EP device, the master control switching unit of the aforementioned RC device can first identify the online EP devices based on their online status, and record them as candidate EP devices. Then, based on the device category of the candidate EP devices, it can find the EP device that uses a SOC chip or SOC device as the primary component, which will be the final first master EP device. All other EP devices besides the first master EP device can act as non-master EP devices and perform business functions. Optionally, if multiple EP devices using a SOC chip or SOC device are found among the candidate EP devices, they can be further filtered according to their device IDs, for example, selecting the EP with the smallest device ID as the final first master EP device.

[0063] Optionally, when the first master control EP device malfunctions, the master control switching unit of the aforementioned RC device can continue to filter out a second master control EP device from the non-master control EP devices based on any one or more combinations of the online status, device category, and device ID of the non-master control EP devices. Of course, a third master control EP device can also be filtered out when the second master control EP device malfunctions again, and so on. The process can be found in the second master control EP device filtering process, which will not be elaborated here.

[0064] In this embodiment, the master control switching unit of the RC device can first filter out online EP devices based on their online status, and then select EP devices of the first category as the first master control EP devices based on their device categories. This two-level filtering process simplifies the complexity of master control EP device selection and improves the accuracy and efficiency of the selection. Furthermore, EP devices capable of performing master control functions are those using SOC chips, while those not capable of performing master control functions are those using non-SOC chips. This facilitates the execution of master control functions by EP devices capable of performing them. Moreover, the multiple EP devices include at least two EP devices capable of performing master control functions, enabling multiple master control functions and thus improving system stability and reliability.

[0065] The following examples illustrate a system that includes multiple levels of bridging devices.

[0066] Figure 4 This is the second architecture diagram of the PCIe bus-based control system provided in this embodiment of the invention. See also... Figure 4 As shown, the bridging device includes a first-level bridging device and at least one second-level bridging device, wherein the second-level bridging device is connected to the first-level bridging device.

[0067] The first-level bridging device is directly connected to the RC device and can be denoted as SWITCH1. The second-level bridging device can be denoted as SWITCH2. There can be one or more second-level bridging devices. The second-level bridging device can be directly connected to the first-level bridging device or indirectly connected through some of the second-level bridging devices.

[0068] In addition, the second-level bridging device is connected to the first-level bridging device in a pluggable manner, such as through a backplane, PCIe slot, or cable connection.

[0069] Each of the aforementioned second-level bridging devices can connect to multiple EP devices. For example, a SWITCH2 shown in the figure can be connected to SWITCH1 and simultaneously connect to multiple EP devices EP6 to EP12. Each of the multiple EP devices EP6 to EP12 can be an EP device primarily using a SOC chip or SOC device, or an EP device primarily using a non-SOC chip or non-SOC device.

[0070] Furthermore, the aforementioned RC device can be placed on the same hardware board as the first-level bridging device SWITCH1. For example, the RC device and the first-level bridging device SWITCH1 can be soldered onto the same hardware board. This shortens the distance between the RC device and the first-level bridging device SWITCH1, meaning they are placed closer together, which avoids system performance loss and further enhances system reliability. Alternatively, the aforementioned RC device and the first-level bridging device SWITCH1 can be connected using devices with good conductivity, such as metals, to further enhance system reliability.

[0071] In this embodiment, the bridging device includes a first-level bridging device and at least one second-level bridging device. The second-level bridging device is connected to the first-level bridging device, which allows for the connection of more EP devices, enabling a larger number of master control devices and enhancing system reliability and stability.

[0072] Based on the PCIe bus-based control system described in the above embodiments, the following embodiments illustrate the PCIe bus-based control method applied to the system.

[0073] Figure 5This is a flowchart illustrating the control method based on the PCIe bus provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the method includes the following steps: Step 502: The bridging device connects to the RC device and each EP device via the PCIE bus.

[0074] Step 504: The RC device initializes the PCIE bus-based control system, acquires the device information of each EP device, determines the first master control EP device and non-master control EP devices among the multiple EP devices based on the device information of each EP device, and sends a first control command to the first master control EP device; the first control command is used to instruct the first master control EP device to switch from performing business functions to master control functions.

[0075] Step 506: The first master control EP device switches itself from business function to master control function according to the first control instruction and executes the master control function; the master control function is used to manage all devices connected to the control system based on the PCIe bus.

[0076] Step 508: The non-master control EP device executes the corresponding service functions according to the service configuration information.

[0077] The specific functions performed by the bridging device, RC device, first master control EP device, and non-master control EP device can be found in the explanations and descriptions in the above system implementation examples, and will not be repeated here.

[0078] In this embodiment, the PCIe bus-based control system includes an RC device, multiple EP devices, and a bridging device. The bridging device connects to the RC device and each EP device via the PCIe bus. The RC device initializes the PCIe bus-based control system, acquires device information from each EP device, determines a first master EP device and non-master EP devices based on this information, and sends a first control command to the first master EP device. The first master EP device switches from a business function to a master function based on the first control command and executes the master function. Non-master EP devices execute corresponding business functions based on their business configuration information. The first control command instructs the first master EP device to switch from a business function to a master function. The master function manages all devices connected to the PCIe bus-based control system. In this method, since the RC device can select a master device from among the EP devices using their device information, each EP device can potentially become a master device. Therefore, a multi-master function can be implemented in the PCIe bus-based control system, and the number of master devices is determined by the number of EP devices, thus supporting a larger number of master devices. Meanwhile, when the main control EP device is functioning normally, other EP devices can be configured to perform service functions through service configuration information, which can achieve higher performance.

[0079] In some embodiments, the above method further includes: The aforementioned RC device detects whether the first master control EP device is abnormal, and if it is determined that the first master control EP device is abnormal, it determines the second master control EP device among the non-master control EP devices based on the device information of each EP device, and sends a second control command to the second master control EP device; the aforementioned second control command is used to instruct the second master control EP device to perform a function switch from business function to master control function; The aforementioned RC device is also used to receive and store the current system configuration information and service configuration information transmitted by the first master control EP device; The aforementioned second master control EP device is used to switch itself from business function to master control function according to the second control command, obtain system configuration information and business configuration information from the RC device, and execute master control function according to the system configuration information and business configuration information; The aforementioned RC device is also used to, in the event that the second master control EP device is found to be abnormal, identify a third master control EP device among the non-master control devices other than the second master control EP device, designate the third master control EP device as the new second master control EP device, and return to execute the aforementioned step of sending the second control command to the second master control EP device.

[0080] For detailed explanations of this embodiment, please refer to the explanations of the above system embodiment; they will not be repeated here.

[0081] In some embodiments, the device information of the EP device includes the online status and device category of the EP device, wherein the device category includes a first category or a second category; the step 504 above, which determines the first master EP device and non-master EP devices among multiple EP devices based on the device information of each EP device, may include: Based on the online status of each EP device, candidate EP devices are determined from among all EP devices; candidate EP devices are those whose online status is online. Based on the equipment category of the candidate EP devices, a first master control EP device is determined among the candidate EP devices, and the other EP devices in each EP device are determined as non-master control EP devices; the aforementioned first master control EP device is a candidate EP device whose equipment category is the first category.

[0082] For detailed explanations of this embodiment, please refer to the explanations of the above system embodiment; they will not be repeated here.

[0083] The following is an overall embodiment to illustrate the control method based on the PCIe bus of this invention. See also: Figure 6 The diagram shown illustrates a specific flow of a control method based on the PCIe bus. This method may include the following steps: Upon system startup, the PCIE enumeration unit in the RC device performs system enumeration, initializing the entire PCIE bus-based control system (referred to as the PCIE system or system). The configuration storage unit in the RC device saves the system configuration information / related information of the PCIE system and obtains the online status / online condition / presence status and device category of each EP device (assuming EP0~EPn, n>1). Then, based on the online status and device category of each EP device, the RC device assumes that EP0 is selected as the current master EP device and maps all / all interrupt routines of the system to EP0.

[0084] Afterwards, all EP devices execute the main control function or service function switch according to the control command sent by the RC device. Then, the main control EP device sends the obtained service configuration information to the RC device for storage. When the main control device is switched in the future, the newly switched main control EP device can read the service configuration information from the RC device.

[0085] Both the master control EP device and the RC device detect whether the current master control EP device is malfunctioning. If no malfunction occurs, the current master control EP device continues to perform the master control function. If the current master control EP device is malfunctioning, the malfunctioning master control EP device will stop feeding the watchdog timer. The RC device will detect the malfunction and then select a new EP device as the new master control EP device based on the device category of the other online EP devices. It will then remap all / all interrupt routines of the system to the new EP device and continue to execute the steps of switching master control functions or service functions according to the control commands sent by the RC device.

[0086] In this embodiment, when the current master control EP device is functioning normally, the PCIe system can achieve high performance by configuring other devices to handle service functions. When the current master control EP device malfunctions, the master control device can be dynamically and flexibly configured to ensure normal system operation and enhance system reliability. Furthermore, the system can have more than two master control devices, allowing for any number of master control devices, thus further increasing system reliability even at the cost of some performance.

[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control system based on a PCIe bus, characterized in that, include: Root endpoint RC device, multiple endpoint EP device, and bridging device; The bridging device is used to connect to the RC device and each of the EP devices via the PCIE bus; The RC device is used to perform system initialization on the PCIE bus-based control system. The RC device is further configured to acquire device information of each EP device, determine a first master control EP device and a non-master control EP device among multiple EP devices based on the device information of each EP device, and send a first control command to the first master control EP device; the first control command is used to instruct the first master control EP device to switch from performing a business function to a master control function. The first master control EP device is used to switch itself from business function to master control function according to the first control command, and execute the master control function; the master control function is used to manage all devices connected to the PCIe bus-based control system; The non-master control EP device is used to execute corresponding business functions according to the business configuration information.

2. The control system based on the PCIe bus according to claim 1, characterized in that, The RC device is also used to detect whether the first master control EP device is abnormal, and if it is determined that the first master control EP device is abnormal, to determine the second master control EP device from the non-master control EP devices according to the device information of the non-master control EP devices, and to send a second control command to the second master control EP device; the second control command is used to instruct the second master control EP device to perform a function switch from business function to master control function; The RC device is also used to receive and store the current system configuration information and service configuration information transmitted by the first master control EP device; The second master control EP device is used to switch itself from business function to master control function according to the second control command, and to obtain the system configuration information and the business configuration information from the RC device, and to execute the master control function according to the system configuration information and the business configuration information; The RC device is further configured to, when it is determined that the second master control EP device is abnormal, identify a third master control EP device among the non-master control devices other than the second master control EP device, designate the third master control EP device as the new second master control EP device, and return to the step of sending the second control command to the second master control EP device.

3. The control system based on the PCIe bus according to claim 1 or 2, characterized in that, The RC device includes a PCIE enumeration unit, an interrupt remapping unit, a configuration storage unit, a master control watchdog unit, and a master control switching unit; The PCIE enumeration unit is used to perform system initialization on the PCIE bus-based control system. The interrupt remapping unit is used to map the interrupt program of the PCIe bus-based control system to the master control EP device currently executing the master control function; the master control EP device includes any one of the first master control EP device, the second master control EP device, and the third master control EP device; The configuration storage unit is used to receive and store the current system configuration information and service configuration information of the PCIe bus-based control system. The system configuration information includes the allocation of master control EP devices and non-master control EP devices and the functional status of the master control EP devices. The service configuration information is reported by the master control EP devices to the RC devices and includes the configuration information of the service functions to be executed by the non-master control EP devices. The main control watchdog unit is used to monitor whether each EP device is online, obtain the online status of each EP device, and obtain the device information of each EP device based on the online status of each EP device. The master control switching unit is used to determine the master control EP device and the non-master control EP device, and to send control commands to the master control EP device; the control commands include the first control command or the second control command.

4. The control system based on the PCIe bus according to claim 3, characterized in that, The device information of the EP device includes the online status and device category of the EP device, and the device category includes a first category or a second category; The main control switching unit is specifically used to determine candidate EP devices among the EP devices based on the online status of each EP device; the candidate EP devices are EP devices that are online. The first master control EP device is determined from the candidate EP devices according to the device category of the candidate EP devices, and the other EP devices in each EP device except the first master control EP device are determined as the non-master control EP devices; the first master control EP device is a candidate EP device whose device category is the first category.

5. The control system based on the PCIe bus according to claim 4, characterized in that, The first category is used to indicate that the EP device is an EP device that uses a system-on-a-chip (SoC) chip, and the second category is used to indicate that the EP device uses a non-SoC chip.

6. The control system based on the PCIe bus according to claim 4, characterized in that, The plurality of EP devices includes at least two EP devices belonging to the first category.

7. The control system based on the PCIe bus according to claim 1, characterized in that, The bridging device includes a first-level bridging device and at least one second-level bridging device, wherein the second-level bridging device is connected to the first-level bridging device.

8. A control method based on a PCIe bus, characterized in that, Applied to a PCIe bus-based control system as described in any one of claims 1 to 7, the method comprises: The bridging device is connected to the RC device and each of the EP devices via the PCIE bus; The RC device initializes the PCIE bus-based control system, acquires device information of each EP device, determines a first master EP device and non-master EP devices among multiple EP devices based on the device information of each EP device, and sends a first control command to the first master EP device; the first control command is used to instruct the first master EP device to switch from performing business functions to master functions. The first master control EP device switches itself from business function to master control function according to the first control command, and executes the master control function; the master control function is used to manage all devices connected to the PCIe bus-based control system; The non-master control EP device executes the corresponding service functions according to the service configuration information.

9. The control method based on the PCIe bus according to claim 8, characterized in that, The method further includes: The RC device detects whether the first master control EP device is abnormal, and if it is determined that the first master control EP device is abnormal, it determines the second master control EP device among the non-master control EP devices according to the device information of the non-master control EP devices, and sends a second control command to the second master control EP device; the second control command is used to instruct the second master control EP device to perform a function switch from business function to master control function; The RC device receives and stores the current system configuration information and service configuration information transmitted by the first master control EP device; The second master control EP device switches itself from service function to master control function according to the second control command, and obtains the system configuration information and the service configuration information from the RC device, and executes the master control function according to the system configuration information and the service configuration information; If the RC device determines that the second master control EP device is abnormal, it identifies a third master control EP device among the non-master control devices other than the second master control EP device, uses the third master control EP device as the new second master control EP device, and returns to the step of sending the second control command to the second master control EP device.

10. The control method based on the PCIe bus according to claim 8, characterized in that, The device information of the EP device includes the online status and device category of the EP device, wherein the device category includes a first category or a second category; the step of determining the first master EP device and non-master EP devices among multiple EP devices based on the device information of each EP device includes: Based on the online status of each EP device, candidate EP devices are determined from among the EP devices; the candidate EP devices are those that are online. The first master control EP device is determined from the candidate EP devices according to the device category of the candidate EP devices, and the other EP devices in each EP device except the first master control EP device are determined as the non-master control EP devices; the first master control EP device is a candidate EP device whose device category is the first category.

Images