Intelligent redundancy implementation method for host and standby based on PCIE (Peripheral Component Interface Express) architecture

Abstract

The invention relates to the technical field of PCIE (Peripheral Component Interconnect Express) communication, and provides a PCIE architecture-based intelligent redundancy realization method for a host and a standby machine, which can realize communication between the host and the standby machine and PCIE peripheral equipment, can realize PICE interface hot standby of the host and the standby machine by taking an FPGA (Field Programmable Gate Array) as a platform, can also realize real-time switching control of the host and the standby machine, and meets PCIE communication of two host and one slave of a system. The PCIE communication between the main and standby CPUs and the peripheral equipment is realized, the PCIE communication between the hot backup and the peripheral equipment is realized, after one CPU fails, the other CPU can be quickly switched to realize the communication with the peripheral equipment, and the switching time is within 50ms.

Description

Technical Field

[0001] The present invention relates to the technical field of PCIE communication, and particularly relates to a method for realizing intelligent redundancy of a primary and standby machine based on a PCIE architecture. Background Art

[0002] The PCIE bus was proposed by Intel Corporation and is the third-generation high-performance I / O bus used to interconnect a computer and peripheral devices. It often uses the x1, x2, x4, and x8 methods to connect to the CPU, FPGA, and peripheral devices. The PCIE bus has the characteristics of high reliability, high speed, point-to-point serial connection, half-duplex transmission, differential link, etc. The PCIE bus has evolved from PCIE 1.0 to the current PCIE 5.0, and the single-channel maximum transmission rate has reached 32 Gbps and is widely used in the field of embedded systems.

[0003] Since the PCIE interface communicates based on the point-to-point and master-slave structure, when the primary and standby CPUs are connected to a single-channel PCIE peripheral device, the traditional redundancy design method is to use a PCIE switch chip to realize communication between multiple PCIE bus devices. However, when the current PCIE switch chip realizes two-master-one-slave communication, the primary and standby CPUs cannot both be used as the host at the same time. Only one CPU can be used as the main control port, and the other CPU is used as the standby port. When the host CPU fails, the device needs to be reset and then enumerated again for communication, and the standby CPU can switch to the main control port. The switching time cannot be guaranteed, and most exceed 100 ms. Especially when starting the operating system, the CPU startup time is longer, and the switching time can reach dozens of seconds, which cannot meet the real-time requirement of CPU hot backup switching. Summary of the Invention

[0004] In view of this, the present invention provides a method for realizing intelligent redundancy of a primary and standby machine based on a PCIE architecture, which can realize communication between the primary and standby machines and PCIE peripheral devices. Based on an FPGA platform, it can not only realize hot backup of the PICE interfaces of the primary and standby machines but also realize real-time switching control of the primary and standby machines, meeting the two-master-one-slave PCIE communication of the system.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A method for realizing intelligent redundancy of a primary and standby machine based on a PCIE architecture includes the following steps:

[0007] Configure at least one PCIE IP core inside the FPGA in the root complex mode, and configure at least two PCIE IP cores inside the FPGA in the endpoint mode. All PCIE IP cores are set to the x1 mode;

[0008] Set up a heartbeat frame mechanism, where the host and the standby machine send heartbeat information to the FPGA through the PCIE interface at a fixed period, and the FPGA monitors and analyzes the heartbeat information status indication signals of the host and the standby machine;

[0009] Set up multiple counters in the FPGA to monitor the delay time and control the timing and clearing according to the heartbeat information status indication signals;

[0010] When the device is powered on or reset, the initialization process starts. Check whether the handshake between the host or the standby machine is successful. If successful, set the corresponding machine to 1 to indicate that the initialization is completed; otherwise, continue to reset the corresponding machine until the handshake is successful;

[0011] The system defaults to giving priority to the host for operation. When it detects that the host CPU is abnormal, that is, no heartbeat information is received within the preset time, the FPGA switches the data forwarding channel to the standby machine CPU, and the switching time is less than 30ms;

[0012] Similarly, when the standby machine CPU is detected as abnormal, the FPGA will switch the data forwarding channel back to the host CPU within the preset time, and also ensure that the switching time is less than 30ms;

[0013] Reset and restart the failed CPU. If the heartbeat information still cannot be restored after three consecutive restarts, do not attempt to reset this CPU anymore.

[0014] Among them, the host and the standby machine are powered on and working simultaneously, establish a PCIE link with the FPGA, complete the initialization operation, and ensure the success of the hot standby of the host and the standby machine;

[0015] The FPGA monitors the heartbeat information periodically sent by the host and the standby machine in real time to control the switching between the host and the standby machine.

[0016] Among them, a two-master-one-slave PCIE interface communication architecture is adopted. After one of the CPUs fails, it quickly switches to the other one to maintain communication with the peripheral devices; other functions are extended through the programmable characteristics of the FPGA.

[0017] Among them, at least one PCIE IP core configured in the RC mode and at least two PCIE IP cores configured in the EP mode are used to establish a PCIE connection with the host and the standby machine; a heartbeat information monitoring module is used to receive and analyze the heartbeat information from the host and the standby machine.

[0018] Among them, the counter module includes at least four counters to monitor the delay time and control the timing and clearing according to the heartbeat information status indication signals; judge and execute the switching between the host and the standby machine according to the heartbeat information status to ensure that the switching time does not exceed 30ms; reset and restart the failed CPU, and limit the number of restarts to prevent meaningless cyclic resets.

[0019] Beneficial effects:

[0020] 1. The method of the present invention realizes the PCIE communication between the primary and standby CPUs and peripheral devices to achieve hot backup for the PCIE communication with peripheral devices. After one of the CPUs fails, it can quickly switch to the other CPU to realize communication with peripheral devices, and the switching time is within 50 ms.

[0021] 2. Compared with the PCIE switch chip, when the present invention method realizes the online switching of two primary and standby machines, the primary and standby switching is realized through the FPGA, and the switching time is not restricted by the startup and restart reset of the CPU, improving the reliability of the system.

[0022] 3. In the method of the present invention, the PCIE communication is realized through the FPGA, which is not limited to only configuring 2 EPs and 1 RC for communication, and can also be configured with 3 EPs or 3 RCs for communication according to needs to meet the requirements of PCIE communication.

[0023] 4. The hardware link adopted by the method of the present invention is not limited to the x1 connection mode for communication, and can also be used for x2, x4, x8 connection modes for communication. The communication rate is not limited to 2.5 Gbps communication, and can also be configured for 5 Gbps and 8 Gbps communication.

[0024] 5. The method of the present invention can be extended to enable the primary and standby CPU to communicate with a single-channel PCIE peripheral device at the same time. The non-transparent bridge transmission is realized through the FPGA to conduct time-sharing communication with the peripheral device.

[0025] 6. The present invention utilizes the programmable design feature of the FPGA, and the FPGA can also expand other functions to realize the function of multi-purpose use of one chip, which not only reduces the hardware design cost but also improves the reusable rate of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the PCIE bus interconnection structure in the embodiment of the present invention.

[0027] Figure 2 It is a schematic diagram of the PCIE interface connection between the CPU and peripheral devices in the embodiment of the present invention.

[0028] Figure 3 It is a schematic diagram of the intelligent redundant switching workflow of the primary and standby machines based on PCIE in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The following takes embodiments in conjunction with the drawings to describe the present invention in detail.

[0030] The present invention provides a method for realizing intelligent redundancy of a primary and standby machine based on the PCIE architecture, which realizes the communication between the primary and standby machines and PCIE peripheral devices. Based on the FPGA platform, it can not only realize the hot standby of the PCIE interfaces of the primary and standby machines, but also realize the real-time switching control of the primary and standby machines, meet the PCIE communication of two masters and one slave in the system, solve the problem that the switching time of the current PCIE switch chip cannot meet the system communication, and improve the reliability of the system.

[0031] For the redundant design hardware platform of the redundant implementation method in this embodiment, the FPGA JFM7VX690T of the JFM7 series of Shanghai Fudan Microelectronics is selected. There are 3 PCIE hard cores integrated inside this chip. This IP core supports four link channels of x1, x2, x4, and x8. There are 3 transmission rates available for single-channel transmission, which are 2.5Gbps, 5Gbps, and 8Gbps respectively. The reference clocks available for selection are 100Mhz, 125Mhz, and 250Mhz respectively. This IP core supports two modes: Root Complex (RC) and Endpoint (EP), which can be set according to system requirements. The main components of the PCIE bus include: Root Complex, Switch, Endpoint Device, and PCI Express to PCI Bridge, and its structure is as follows Figure 1 shown. Among them, the Root Complex connects the CPU and memory to the PCI Express switching structure composed of one or more switching devices; the Switch is a device with 2 or more ports, and each port can lead out a PCIE link for the interconnection of multiple devices; the Endpoint Device is a specific functional device, and only one Endpoint Device can be hung on each link; the PCIE to PCI Bridge device realizes the communication protocol conversion between PCIE and PCI.

[0032] Taking the intelligent redundant design of the primary and standby CPUs based on the PCIE interface as an example, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0033] The host, standby machine, and peripheral devices establish communication through the PCIE interface. The primary and standby machines adopt the hot standby method. When the host fails, it can quickly switch to the standby machine to work. During the entire switching process, the system does not need to be powered on or reset, thus affecting the operation of the system.

[0034] The intelligent redundant design method of the primary and standby machines based on PCIE of the present invention realizes non-transparent bridge transmission through the FPGA. The system is designed for two masters and one slave communication. The purpose is that when the primary and standby machines switch, it can be completed instantaneously, thus not affecting system communication. This design method is not only limited to the PCIE protocol, but can also be used for the redundant design of primary and standby machines in the RapidIO protocol and network protocol. In this embodiment, taking the usage mode of the intelligent redundant design of the primary and standby machines and the connection mode of the PCIE interface between the host, standby machine, and peripheral devices, such as Figure 2As shown in the figure, through 3 PCIE links, 3 independent PCIE IP modules are set in the FPGA for communication. Taking the heartbeat information as the judgment condition, the master-slave machine switching is controlled, which can not only realize the hot standby of the PICE interfaces of the master-slave CPUs, but also realize the real-time switching control of the master-slave CPUs, meeting the needs of the system CPU redundancy design. The specific implementation steps of this embodiment are as follows:

[0035] (1) Configure 1 PCIE IP core inside the FPGA, set it to RC mode, x1 mode, set the communication rate to 2.5Gbps, and the reference clock to 100Mhz.

[0036] (2) Configure 2 PCIE IP cores inside the FPGA, set them to EP mode, x1 mode, set the communication rate to 2.5Gbps, and the reference clock to 100Mhz. The connection relationship of the entire system is as Figure 2 shown.

[0037] (3) Set the heartbeat frame. The master and slave machines send heartbeat information to the FPGA through the PCIE interface every 5ms cycle. The FPGA monitors the variable values of the heartbeat information status indication signal (A_Work_state) of the master machine and the variable values of the heartbeat information status indication signal (B_Work_state) of the slave machine.

[0038] (4) Inside the FPGA, set 4 counters. Counters A1 and B1 are used for timing delay; the timing and clearing of counters A2 and B2 are independently controlled according to the heartbeat information status indication signal.

[0039] (5) As Figure 3 shown, after the device is powered on or reset, the master machine (A) or the slave machine (B) starts to work, and counters A1 and B1 start to time. After the timing exceeds 30s, it is judged whether LinkA (master machine) or LinkB (slave machine) has successfully held hands (high level is valid). After successfully holding hands, the A machine or the B machine is set to 1, and the initialization is completed; otherwise, the corresponding master or slave machine needs to be reset continuously, and step (5) is continued to be executed.

[0040] (6) By default, when the system is powered on, the master machine works or the system is already working in the master mode, step (7) is executed; if the system has been switched to the slave machine, step (9) is executed.

[0041] (7) Monitor the heartbeat information indication signal (A_Work_state) of the master CPU. If the FPGA does not receive the heartbeat information of the master CPU within 25ms, it means that the master CPU is working abnormally, the A machine is set to 0, and step (8) is executed;

[0042] Otherwise, the master machine works normally, no machine switching is performed, and step (7) is executed.

[0043] (8) If machine A equals 0 (host abnormal) and at this time machine B equals 1 (standby normal), the FPGA stops the interaction between the host CPU and the peripheral devices. The FPGA switches the data forwarding channel to the standby CPU, and the switching time is less than 30 ms, then step (9) is executed; otherwise, the host does not switch machines, resets the host, and step (5) is executed.

[0044] (9) Monitor the heartbeat information indication signal (B_Work_state) of the standby CPU. If the FPGA does not receive the heartbeat information of the standby CPU within 25 ms, it indicates that the standby CPU is working abnormally. Set machine B to 0 and execute step (10);

[0045] Otherwise, the standby is working normally, no machine switching is performed, and step (9) is executed.

[0046] (10) If machine B equals 0 (standby abnormal) and at this time machine A equals 1 (host normal), the FPGA stops the interaction between the standby CPU and the peripheral devices. The FPGA switches the data forwarding channel to the host CPU, and the switching time is less than 30 ms, then step (8) is executed; otherwise, the standby does not switch machines, resets the standby, and continues to execute step (6).

[0047] In this embodiment, in steps (5) to (10) above, for the main and standby machine switching, by judging the heartbeat information status, there are actually 2 heartbeat information indication signals corresponding to the main and standby machines in the actual design. After the device is powered on, the main and standby machines are powered on and work simultaneously, establish a PCIE link with the FPGA, and complete the initialization operation. The main and standby machines are successfully hot standby; the FPGA monitors the heartbeat information periodically sent by the main and standby machines, controls the main and standby machine switching, and the switching time is completed instantaneously, solving the problem of long switching time of the main and standby machines of the PCIE switching chip.

[0048] This embodiment uses the FPGA as a platform. Using the FPGA as a platform, it can not only realize the PICE interface hot standby of the main and standby CPUs, but also realize the real-time switching control of the main and standby CPUs, meeting the needs of the system CPU redundancy design; realizing the main and standby machine redundancy design, establishing a two-master-one-slave PCIE interface communication, and realizing the main and standby machine hot standby function; after one of the CPUs fails, it can quickly switch to the other CPU to realize communication with the peripheral devices, and the switching time requirement is within 30 ms; after switching to the other CPU, at the same time, reset the failed CPU and restart it. If the failed CPU still cannot recover the heartbeat information after restarting 3 times, no more reset operations will be performed. Based on the existing technology, this technical solution utilizes the programmable design characteristics of the FPGA to solve the problem of long switching time of the main and standby machines of the PCIE switching chip. At the same time, the FPGA can expand other functions to realize the function of multi-purpose in one chip, which not only reduces the hardware design cost but also improves the reusable rate of the device.

[0049] In summary, the above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for realizing intelligent redundancy of master and standby machines based on PCIE architecture, characterized in that: The following steps are involved: Configure at least one PCIE IP core in the FPGA as a root complex mode, and configure at least two PCIE IP cores in the FPGA as endpoint mode. All PCIE IP cores are set to x1 mode. Set up a heartbeat frame mechanism, in which the host and the standby send heartbeat information to the FPGA at a fixed period through the PCIE interface, and the FPGA monitors and analyzes the heartbeat information status indication signals of the host and the standby; Multiple counters are set in the FPGA to monitor the delay time and control the timing and clearing according to the heartbeat information status indication signal; When the device is powered on or reset, the initialization process begins to check whether the handshake between the master and backup machines is successful. If successful, the corresponding machine is set to 1 to indicate that the initialization is complete. Otherwise, continue to reset the corresponding machine until the handshake is successful; the system defaults to giving priority to the host machine. When an abnormality is detected in the host CPU, that is, the heartbeat information is not received within the preset time, the FPGA switches the data forwarding channel to the standby CPU, and the switching time is less than 30ms; Similarly, when the standby CPU is detected as abnormal, the FPGA will switch the data forwarding channel back to the host CPU within a preset time, also ensuring that the switching time is less than 30ms; Reset and restart the failed CPU. If the heartbeat information cannot be restored after three consecutive restarts, no more attempts are made to reset the CPU.

2. The method according to claim 1, characterized in that The main and standby machines are powered on at the same time, a PCIE link is established with the FPGA, and the initialization operation is completed to ensure the success of the hot standby of the main and standby machines; The FPGA monitors the heartbeat information periodically sent by the master and standby machines in real time to control the switching between the master and standby machines.

3. The method according to claim 2, characterized in that ,Adopting two master and one slave PCIE interface communication architecture, it quickly switches to another channel when one CPU fails to maintain communication with peripheral devices; and expands other functions through FPGA programmable characteristics.

4. The method according to any one of claims 1 to 3, characterized in that , at least one PCIE IP core configured in RC mode and at least two PCIE IP cores configured in EP mode, used to establish PCIE connections with the host and the standby machine; The heartbeat information monitoring module is used to receive and parse the heartbeat information from the host and the backup machine.

5. The method according to claim 4, characterized in that The counter module includes at least four counters, which are used to monitor the delay time and control the timing and clearing according to the heartbeat information status indication signal; judge and execute the switching between the primary and standby machines according to the heartbeat information status to ensure that the switching time does not exceed 30ms; reset and restart the failed CPU, and limit the number of restarts to prevent meaningless cyclic reset.

Images