Automated Testing Framework for Control Devices Using Grammar-Based Test Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current testing frameworks for computing devices in manufacturing environments are inefficient in recreating inputs and operating environments, leading to inadequate assessment of system performance, especially when devices enter 'fail safe' modes, which can mask errors and vulnerabilities.
Innovation Solution
An automated testing framework that generates test cases using a grammar system to monitor digital inputs and outputs, compares actual outputs to expected results, and uses a fault isolation engine to detect anomalies, operating as a communication bridge between components to assess the robustness and security of process control devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a testing framework communicates solely with the device under test, then the testing setup is simple, but the testing accuracy is insufficient because it cannot properly assess system performance when devices enter fail safe modes
Solution Approach 1:
The patent introduces a testing framework that acts as an intermediary between the device under test and the testing environment. The framework includes a communication bridge that intercepts and analyzes messages between the PLC and other system components, enabling accurate detection of fail-safe mode behavior without requiring direct access to all system components. This mediator approach allows the testing system to assess whether the PLC properly enters fail-safe mode and stops processing control instructions.
2Measurement precision
If the testing framework analyzes inputs and outputs from each computing device in the system, then the testing accuracy improves, but the testing efficiency decreases due to the need to monitor multiple devices
Solution Approach 1:
The patent extracts the essential testing functionality from a complex multi-device monitoring system. Instead of analyzing inputs and outputs from every computing device in the system, the framework focuses on intercepting and analyzing only the critical communication messages between the PLC and the control system. This extraction approach maintains high testing accuracy for fail-safe mode detection while significantly improving testing efficiency by reducing the scope of monitoring to essential communication paths.
3Reliability
If the PLC enters fail safe mode to prevent damage, then the safety is improved, but the ability to detect errors through normal output monitoring is reduced
Solution Approach 1:
The patent implements a feedback mechanism where the testing framework monitors whether the PLC properly enters fail-safe mode and stops processing control instructions. The framework provides feedback by detecting when the PLC transitions to fail-safe mode through analysis of communication messages, and verifies that the PLC maintains this state by refusing to process further instructions. This feedback approach enables continuous error detection capability even when the PLC is in fail-safe mode, as the testing framework can observe the PLC's refusal to process instructions as evidence of proper fail-safe behavior.
Data Source
AI summary
The present disclosure generally relates to the automated testing of a system that includes software or hardware components. In some embodiments, a testing framework generates a set of test cases for a system under test using a grammar. Each test case may perform an action, such as provide an input to the system under test, and result in an output from the system under test. The inputs and outputs are then compared to the expected results to determine whether the system under test is performing correctly. Specifically, the system under test may be analyzed to determine whether it is capable of properly processing control instructions and input signals and/or generating expected output control signals and additional control/feedback information. The data can then be interpreted in the grammar system and/or used as input to a fault isolation engine to determine anomalies in the system under test.