A fault-tolerant verification system and method supporting soft and hard fault injection and voting

Abstract

A fault-tolerant verification system and method supporting software and hardware fault injection and voting, wherein the fault-tolerant verification system supporting software and hardware fault injection and voting comprises fault injection management software, a power management module, a fault injection device, a simulation device and an entity device. The fault-tolerant verification method supporting software and hardware fault injection and voting is performed by means of the above system. A suitable fault model is established, and meanwhile, it is ensured that the faults of the verification system will not cause damage to the real device.

Description

Technical Field

[0001] This invention relates to the field of fault-tolerant system design, and in particular to a fault-tolerant verification system and method that supports software and hardware fault injection and voting. Background Technology

[0002] The fault injection verification system is a crucial functional module of the simulation platform. It can artificially introduce faults into the target system according to a pre-selected fault model and a specific strategy. By observing and analyzing the system's behavior under the injected fault conditions, it provides the necessary qualitative and quantitative evaluation results. The implementation of fault injection involves multiple aspects of the control system, is closely coupled with the design, and crosses many professional fields. It is a highly complex process that requires the use of various technical means to simulate different types of faults from different perspectives.

[0003] In the development of control systems, testing the design and execution of all system functions is crucial. The design must consider not only the normal system state but also various abnormal states. After model acquisition, control law design, and control parameter calibration, ensuring the controller's performance remains stable under changes in external environment or operating conditions, functions normally during sudden general faults, and maintains acceptable performance degradation during sudden severe faults requires fault injection during simulation verification to simulate and resolve these issues, thereby improving reliability. This also promotes research into the control loop's timing and robustness of control performance.

[0004] Currently, fault injection verification systems mainly simulate hardware fault injection through software, including the verification of electrical signal faults and bus faults. Some fault conditions are simulated by changing the controller software. However, the fault condition simulation is insufficient, unsystematic, heavily reliant on the experience level of the developers, and cannot simulate real hardware fault injection. Summary of the Invention

[0005] This invention provides a fault-tolerant verification system and method that supports software and hardware fault injection and voting, in order to overcome the shortcomings of the prior art, establish a suitable fault model, and ensure that the faults of the verification system will not damage the real equipment.

[0006] In order to achieve the objectives of this invention, the following technologies are proposed:

[0007] The present invention provides a fault-tolerant verification system that supports software and hardware fault injection and voting, including fault injection management software, power management module, fault injection device, simulation device and physical device;

[0008] The fault injection management software controls the power supply of the fault injection device, simulation device and physical device through the power management module;

[0009] The fault injection management software sends fault commands to the fault injection device;

[0010] The fault injection management software receives and processes the output results of the physical or simulated equipment after fault injection collected by the fault injection device, compares and analyzes the prediction results of the fault injection device, and stores the comparison results in the database.

[0011] The fault injection device injects hardware faults into the simulation device or physical device via the IO data cable, and injects software faults into the simulation device or physical device via the bus data cable.

[0012] The power management module is used for power supply and power control of fault injection devices, simulation devices, and physical devices;

[0013] Simulation equipment can partially or fully realize the functions of physical equipment, so as to replace physical equipment for fault injection testing;

[0014] The physical equipment is the main equipment for fault injection and testing.

[0015] Furthermore, the fault injection management software uses a relational database to store data on the fault injection process to the fault injection device, including fault definition, fault execution, fault comparison, and result analysis.

[0016] Furthermore, the fault injection device includes a fault injection and voting board and an interface board plugged into the industrial control computer chassis, and the fault injection and voting board and the interface board are plugged into the industrial control computer chassis in the form of PCIe cards;

[0017] It enables hardware simulation injection of single-machine faults, communication faults, and circuit faults in physical or simulated equipment, as well as bus status capture, and supports fault injection control at the physical and protocol layers.

[0018] The hardware component of the fault injection device is responsible for outputting OC command signals and acquiring the status of the physical device.

[0019] It can simulate and inject hardware faults such as single-machine hardware faults, bus load abnormal faults, and circuit faults.

[0020] The software portion of the fault injection device controls the orderly output of faults and the configuration and generation of fault modes;

[0021] It can simulate and inject software to detect single-machine software failures, communication data failures, etc.

[0022] Furthermore, hardware fault injection in the fault injection device includes simulated power short circuits, short circuits to ground, open circuits, short circuits between two signals, loose connections, and leakage faults injected through fault injection and voting board and interface board.

[0023] Furthermore, software fault injection in fault injection devices includes simulating and injecting faults at the bus physical layer, protocol layer, and software layer into physical devices or simulation devices; including but not limited to simulating checksum, word length shortage, and word spacing faults during bus transmission.

[0024] Furthermore, the fault injection and voting board injects faults through the onboard SoC processor by generating three-machine switching and voting logic using an SRAM-type FPGA;

[0025] Fault injection and voting board include:

[0026] Furthermore, the fault injection and voting board includes:

[0027] The fault injection main control module runs in the onboard SoC processor and controls fault injection and comparison of test results according to test requirements.

[0028] The switching logic module includes switching fault injection and three-machine switching logic. According to the three-machine switching protocol, it injects specific faults into the three-machine switching logic to realize fault injection at the protocol and physical layer, simulating faults in three-machine switching.

[0029] The voting logic module includes voting fault injection and three-machine voting logic. Physical layer fault injection is implemented in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, generate corresponding fault modes, and inject them into the FPGA.

[0030] The voting execution module includes an injection software generation module, an injection result comparison module, and a voting execution equivalent;

[0031] The injection software generation module generates a fault injection vector based on the verification requirements to trigger the kernel to simulate a single-machine fault.

[0032] The voting execution module receives the information required for the decision from the three machines, and the remote control terminal OC command module of the interface board outputs the voting control right and the switching clock source control signal to realize the output of simulated voting rights and the acquisition and calculation of voting data;

[0033] The injection result comparison module compares and preprocesses the predicted results of the fault injection device.

[0034] Furthermore, the physical equipment is a standard three-machine thermal fault-tolerant structure;

[0035] The physical device consists of three identical units, three power boards, and a second interface board. The power boards are connected to the units, the units are connected to the second interface board, and the second interface board is connected to the fault injection device.

[0036] Furthermore, the simulation equipment simulates the input and output interfaces of the physical equipment; the simulation equipment includes an industrial control computer power board, an industrial control computer board, a 1553B bus interface, an RS422 interface, an LVDS interface, and an OC interface.

[0037] This invention provides a fault-tolerant verification method that supports software and hardware fault injection and voting, wherein the fault injection management software sends fault commands to the fault injection main control module in the fault injection device.

[0038] The switching logic module and voting logic module in the fault injection device receive fault commands sent by the fault injection main control module.

[0039] The switching logic module, in accordance with the three-machine switching protocol, injects protocol and physical layer faults into the three-machine switching logic and simulates three-machine switching faults.

[0040] The voting logic module implements physical layer fault injection in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, and generates corresponding fault modes.

[0041] The voting execution module obtains the faults of the three-machine switching simulated by the switching logic module, as well as the interface circuit faults, voting logic faults, and corresponding fault modes of the voting interface circuit simulated by the voting logic module.

[0042] The injection software generation module in the voting execution module generates a fault injection vector according to the verification requirements to trigger the kernel to simulate a single-machine fault and generate fault software for three independent CPUs. The software is injected into the three independent CPUs through the monitoring port to control the three CPUs and thus inject faults into any one of the three CPUs.

[0043] The voting execution equivalent in the voting execution module receives the information required for the three-machine decision, and the remote control terminal module of the interface board outputs the voting control right and the switching clock source control signal to realize the output of simulated voting rights and the acquisition and calculation of voting data.

[0044] The advantages of the above technical solution are:

[0045] In summary, this invention designs a fault-tolerant verification system that supports software and hardware fault injection and voting, enabling rapid prototyping, simulation verification, fault simulation, and fault-tolerant strategy voting for relevant system models and tasks. Through fault injection, software and hardware faults are injected into the dual / triple-machine system, including sensor and actuator anomalies, communication interface anomalies, data anomalies, information flow anomalies, signal transmission anomalies, and control timing anomalies. The functionality, reliability, and fault tolerance capability of the core decision module are verified. By observing the decision results of the dual / triple-machine system under fault injection conditions, the correctness and fault tolerance capability of the developed fault-tolerant system and multi-machine decision system are judged. Simultaneously, it ensures that system faults will not damage real equipment. Furthermore, during the design phase of the target system, simulated fault injection can serve as a means of studying system fault behavior because the fault injection verification method accelerates failure and obtains sufficient failure data in a short time for statistical analysis.

[0046] (1) This method uses hardware and software fault injection technology to verify the functionality, reliability and fault tolerance capability of the core decision module. It can evaluate the effectiveness of the fault tolerance mechanism and predict system performance.

[0047] (2) The establishment of the fault injection system has greatly improved the research capabilities for system design and verification, shortened the development cycle, saved resource costs, improved product performance and data, expanded the research on the basic theories related to the system, and enhanced the means of system testing. It is of great significance for improving the development level of products and improving product quality.

[0048] (3) Since the core unit of the fault injection system adopts SRAM-type FPGA, users can port the fault injection software according to the characteristics of the physical equipment, which has strong scalability.

[0049] (4) The platform is highly reusable and can be easily ported to other large-scale system designs. Attached Figure Description

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings.

[0051] Figure 1 The basic framework of the system is shown.

[0052] Figure 2 The framework of the system is shown.

[0053] Figure 3 The framework of the fault injection device is shown. Detailed Implementation

[0054] Example 1

[0055] like Figures 1 to 3 As shown, the fault injection system uses methods such as simulating faults in electrical signal paths and injecting faults into communication protocols to simulate various faults in real systems. Since fault injection is generally directly targeted at the target system, it can avoid the inaccuracies caused by inaccurate model assumptions, as seen in analytical methods.

[0056] Based on this, a fault-tolerant verification system supporting software and hardware fault injection and voting was developed. This system can repeatedly inject fault states into electronic control units. It includes fault injection management software, a power management module, a fault injection device, a simulation device, and a physical device. During development, the functionality, reliability, and fault tolerance of the core decision module of the physical device were verified using these components. Software and hardware fault injections were performed on three individual units of the physical device. The prediction results of the fault injection and voting board under the fault injection state were observed and compared with the decision results of the three developed units to evaluate the correctness and fault tolerance of the developed three-unit redundancy system.

[0057] The fault injection management software controls the power supply of the fault injection device, simulation device, and physical device through the power management module; sends fault commands to the fault injection device; receives and processes the voting results after fault injection into the physical device or simulation device; compares and analyzes the predicted results of the fault injection device; and stores the comparison results in the database.

[0058] Specifically, the computer runs fault injection management software, which controls the fault injection device and sends fault injection commands on one hand, and controls the power supply at each level through a programmable power supply on the other. It also receives and processes the voting results after fault injection from the physical device, compares and analyzes them with the predicted results from the fault injection board, and stores the comparison results in a database. To facilitate further data analysis, the fault injection management software uses a relational database, which stores relevant data about the fault injection process, including definitions, implementation details, and result analysis.

[0059] Fault injection devices inject hardware faults into simulated or physical devices via I / O data cables, and software faults via bus data cables. These devices can inject faults in various modes, including open circuits, short circuits to ground, short circuits to power supplies, and short circuits between controller pins. Automatic fault injection can also be performed using fault injection models. This meets the requirements for fault injection into physical (simulated) devices.

[0060] The fault injection device includes a fault injection and voting board and an interface board, which are plugged into the industrial control computer chassis. The fault injection and voting board and the interface board are plugged into the industrial control computer chassis in the form of PCIe cards. It realizes hardware simulation injection of single-machine faults, communication faults and circuit faults in physical equipment or simulation equipment, as well as bus status capture, and supports fault injection control at the physical layer and protocol layer.

[0061] The hardware component of the fault injection device is responsible for the input and output of fault injection signals and status acquisition. Hardware fault injection in the fault injection device includes simulating power short circuits, short circuits to ground, open circuits, short circuits between two signals, loose connections, and leakage faults through fault injection, voting board, and interface board.

[0062] The software component of the fault injection device controls the orderly output of faults and the configuration and generation of fault modes. It can simulate and inject software faults, such as single-machine software faults and communication data faults. Software fault injection in the fault injection device includes simulating and injecting faults at the bus physical layer, protocol layer, and software layer into physical or simulated devices. This includes, but is not limited to, simulating checksum, word length shortage, and word spacing faults during bus transmission.

[0063] The fault injection device is the core module for fault injection. Controlled by fault injection management software, the device outputs faults in an orderly manner, enabling hardware simulation injection of single-machine faults, communication faults, and circuit faults, as well as bus status capture. It supports physical layer and protocol layer fault injection control. Employing both hardware and software injection techniques, the hardware handles signal input / output and status acquisition, while the software configures and generates fault modes. It possesses unified management capabilities for multiple faults and types, enabling precise, flexible, and wide-ranging verification, evaluation, and testing of the system.

[0064] Hardware fault injection simulates power short circuits, short circuits to ground, open circuits, short circuits between two signals, loose connections, and leakage through a fault injection board and interface board. It enables autonomous power switching and clock source switching control, supports physical device interface output and data acquisition capabilities, and verifies output validity.

[0065] The software fault injection system can simulate faults in the physical layer, protocol layer, and software layer of the bus system and its components. It supports faults in any single machine / processor in a three-machine redundant computer and can simulate faults such as parity, short word length, and word spacing during bus transmission.

[0066] The entire fault injection process is controlled by fault injection management software. Physically, this software can be either a program running on the target system itself or a separate host machine (such as a computer within this system). It interacts with the target system through a communication interface or other means.

[0067] Among them, the aforementioned fault injection and voting board replaces the original three-machine switching and voting logic with an SRAM-type FPGA, and injects faults through the onboard SoC processor.

[0068] The fault injection and voting board includes: a fault injection main control module, a switching logic module, a voting logic module, and a voting execution module.

[0069] The fault injection main control module runs on the onboard SoC processor and controls fault injection and test result comparison according to test requirements. It implements hardware simulation injection of single-machine faults, communication faults, and circuit faults, as well as bus status capture, supporting physical layer and protocol layer fault injection control. Employing hardware and software injection technology, the hardware part handles signal input / output and status acquisition, while the software part configures and generates fault modes. It possesses unified management capabilities for multiple faults and types, enabling precise, flexible, and wide-ranging verification, evaluation, and testing of the system.

[0070] The switching logic module includes switching fault injection and three-machine switching logic. According to the three-machine switching protocol, it injects specific faults into the three-machine switching logic to realize fault injection at the protocol and physical layer, simulating faults in three-machine switching.

[0071] The voting logic module includes voting fault injection and three-machine voting logic. Physical layer fault injection is implemented in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, generate corresponding fault modes, and inject them into the FPGA.

[0072] The voting execution module includes an injection software generation module, an injection result comparison module, and a voting execution equivalent. The injection software generation module generates a fault injection vector based on verification requirements and injects a "lightweight kernel" into the physical device to trigger a kernel-simulated single-machine fault. The voting execution equivalent receives information required for the three-machine decision-making process and outputs voting control rights and clock source control signals from the remote control terminal module on the interface board to achieve the output of simulated voting rights and the acquisition and calculation of voting data. It can generate fault software for three independent CPUs and inject the software into each of the three independent CPUs through the monitoring port, thereby controlling the three CPUs and enabling fault injection into any one of them.

[0073] The injection result comparison module compares and preprocesses the predicted results of the fault injection device.

[0074] The power management module is used for power supply and power control of fault injection devices, simulation devices, and physical devices.

[0075] Simulation equipment can partially or fully realize the functions of physical equipment, allowing for fault injection testing in place of physical equipment. The simulation equipment mimics the input / output interfaces of the physical equipment. The simulation equipment includes an industrial control computer power board, an industrial control computer board, a 1553B bus interface, an RS422 interface, an LVDS interface, and an OC interface.

[0076] The physical device is the main equipment for fault injection and testing. It is a standard three-machine thermal fault-tolerant structure. The physical device consists of three identical individual machines, three power supply boards, and one second interface board. The power supply boards are connected to the individual machines, and the individual machines are all connected to the second interface board. The second interface board is connected to the fault injection and voting board of the fault injection device.

[0077] Example 2

[0078] A fault-tolerant verification method that supports software and hardware fault injection and voting, wherein the fault injection management software sends fault commands to the fault injection main control module in the fault injection device;

[0079] The switching logic module and voting logic module in the fault injection device receive fault commands sent by the fault injection main control module.

[0080] The switching logic module, in accordance with the three-machine switching protocol, injects protocol and physical layer faults into the three-machine switching logic and simulates three-machine switching faults.

[0081] The voting logic module implements physical layer fault injection in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, and generates corresponding fault modes.

[0082] The voting execution module obtains the faults of the three-machine switching simulated by the switching logic module, as well as the interface circuit faults, voting logic faults, and corresponding fault modes of the voting interface circuit simulated by the voting logic module.

[0083] The injection software generation module in the voting execution module generates a fault injection vector according to the verification requirements to trigger the kernel to simulate a single-machine fault and generate fault software for three independent CPUs. The software is injected into the three independent CPUs through the monitoring port to control the three CPUs and thus inject faults into any one of the three CPUs.

[0084] The voting execution equivalent in the voting execution module receives the information required for the three-machine decision, and the remote control terminal module of the interface board outputs the voting control right and the switching clock source control signal to realize the output of simulated voting rights and the acquisition and calculation of voting data.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A fault-tolerant verification method supporting software and hardware fault injection and voting, characterized in that, The verification system includes fault injection management software, power management module, fault injection device, simulation device and physical device; The fault injection management software controls the power supply of the fault injection device, simulation device and physical device through the power management module; The fault injection management software sends fault commands to the fault injection device; The fault injection management software receives and processes the output results of the physical or simulated equipment after fault injection collected by the fault injection device, compares and analyzes the prediction results of the fault injection device, and stores the comparison results in the database. The fault injection device injects hardware faults into the simulation device or physical device via the IO data cable, and injects software faults into the simulation device or physical device via the bus data cable. The power management module is used for power supply and power control of fault injection devices, simulation devices, and physical devices; Simulation equipment can partially or fully realize the functions of physical equipment, so as to replace physical equipment for fault injection testing; The physical equipment is the main equipment for fault injection and testing; in, The fault injection device includes a fault injection and voting board and an interface board that are plugged into the industrial control computer chassis. The fault injection and voting board and the interface board are plugged into the industrial control computer chassis in the form of PCIe cards. It enables hardware simulation injection of single-machine faults, communication faults, and circuit faults in physical or simulated equipment, as well as bus status capture, and supports fault injection control at the physical and protocol layers. The hardware component of the fault injection device is responsible for outputting OC command signals and acquiring the status of the physical device. It can simulate and inject hardware faults, bus load abnormalities, and circuit faults. The software portion of the fault injection device controls the orderly output of faults and the configuration and generation of fault modes; It can simulate and inject software faults and communication data faults. The fault injection and voting board generates three-machine switching and voting logic through an SRAM-type FPGA and injects faults through the onboard SoC processor. Fault injection and voting board include: The fault injection main control module runs in the onboard SoC processor and controls fault injection and comparison of test results according to test requirements. The switching logic module includes switching fault injection and three-machine switching logic. According to the three-machine switching protocol, it injects specific faults into the three-machine switching logic to realize fault injection at the protocol and physical layer, simulating faults in three-machine switching. The voting logic module includes voting fault injection and three-machine voting logic. Physical layer fault injection is implemented in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, generate corresponding fault modes, and inject them into the FPGA. The voting execution module includes an injection software generation module, an injection result comparison module, and a voting execution equivalent; The injection software generation module generates a fault injection vector based on the verification requirements to trigger the kernel to simulate a single-machine fault. The voting execution module receives the information required for the decision from the three machines, and the remote control terminal OC command module of the interface board outputs the voting control right and the switching clock source control signal to realize the output of simulated voting rights and the acquisition and calculation of voting data; The injection result comparison module compares and preprocesses the predicted results of fault injection devices; The methods include: The fault injection management software sends fault commands to the fault injection main control module in the fault injection device; The switching logic module and voting logic module in the fault injection device receive fault commands sent by the fault injection main control module. The switching logic module, in accordance with the three-machine switching protocol, injects protocol and physical layer faults into the three-machine switching logic and simulates three-machine switching faults. The voting logic module implements physical layer fault injection in the three-machine voting circuit to simulate voting interface circuit faults and voting logic faults, and generates corresponding fault modes. The voting execution module obtains the faults of the three-machine switching simulated by the switching logic module, as well as the interface circuit faults, voting logic faults, and corresponding fault modes of the voting interface circuit simulated by the voting logic module. The injection software generation module in the voting execution module generates a fault injection vector according to the verification requirements to trigger the kernel to simulate a single-machine fault and generate fault software for three independent CPUs. The software is injected into the three independent CPUs through the monitoring port to control the three CPUs and thus inject faults into any one of the three CPUs. The voting execution equivalent in the voting execution module receives the information required for the three-machine decision, and the remote control terminal module of the interface board outputs the voting control right and the switching clock source control signal to realize the output of simulated voting rights and the acquisition and calculation of voting data.

2. The fault-tolerant verification method supporting software and hardware fault injection and voting according to claim 1, characterized in that, The fault injection management software uses a relational database to store data on the fault injection process into the fault injection device, including fault definition, fault execution, fault comparison, and result analysis.

3. The fault-tolerant verification method supporting software and hardware fault injection and voting according to claim 1, characterized in that, Hardware fault injection in the fault injection device includes simulated power short circuits, short circuits to ground, open circuits, short circuits between two signals, loose connections, and leakage faults injected through the fault injection and voting board and interface board.

4. The fault-tolerant verification method supporting software and hardware fault injection and voting according to claim 1, characterized in that, Software fault injection in fault injection devices includes simulating and injecting faults at the bus physical layer, protocol layer, and software layer into physical devices or simulation devices; including but not limited to simulating check faults, short word length, and word gap faults during bus transmission.

5. The fault-tolerant verification method supporting software and hardware fault injection and voting according to claim 1, characterized in that, The physical equipment is a standard three-machine thermal fault-tolerant structure; The physical device consists of three identical units, three power boards, and a second interface board. The power boards are connected to the units, the units are connected to the second interface board, and the second interface board is connected to the fault injection device.

6. The fault-tolerant verification method supporting software and hardware fault injection and voting according to claim 1, characterized in that, The simulation equipment simulates the input and output interfaces of physical equipment; the simulation equipment includes an industrial control computer power board, an industrial control computer board, a 1553B bus interface, an RS422 interface, an LVDS interface, and an OC interface.

Images