Multi-layer Test Suite Generation for Operating Platform Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing operating platforms face inconsistencies due to vendor extensions, making it difficult for providers to maintain a consistent user experience and for application developers to create compatible applications across different vendor implementations.
Innovation Solution
A method is introduced to create a test suite that separates the system layer from the framework layer, injecting software faults into the system layer to identify and flag faults, and iteratively adding test cases to the suite until a threshold is satisfied, allowing for comprehensive testing of vendor-specific implementations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If vendor extensions are applied to the operating platform, then vendor-specific functionality is provided, but inconsistencies across vendor implementations are created
Solution Approach 1:
The patent divides the operating platform into distinct layers: a system layer that remains consistent across all vendors and a framework layer that allows vendor-specific extensions. This segmentation enables vendors to customize functionality while maintaining a stable, consistent base platform that ensures uniform behavior and compatibility.
2Reliability
If a compatibility test suite is applied to vendor devices, then consistency of user experience is improved, but the complexity of testing increases
Solution Approach 1:
The test suite is segmented into two distinct parts: system layer tests that verify consistent behavior across all vendors, and framework layer tests that accommodate vendor-specific variations. This segmentation simplifies the testing process by focusing system layer tests on invariant properties while allowing framework layer tests to handle diversity.
Solution Approach 2:
The patent extracts the core consistency verification into separate system layer tests, isolating them from vendor-specific framework layer tests. This extraction allows the consistency testing to be performed independently and more efficiently, reducing the overall complexity by separating concerns.
3Difficulty of detecting and measuring
If fault injection is performed on the system layer, then fault localization is improved, but the complexity of fault detection increases
Solution Approach 1:
The fault detection process is segmented by testing the system layer independently from the framework layer. By injecting faults into the system layer and executing tests that isolate system layer behavior, the patent enables precise fault localization without the interference of vendor-specific framework code, thereby reducing detection complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for developing a multi-layered test suite for an operating platform including a framework layer and a system layer includes executing a first test suite against a version of the operating platform modified based on a software faults. A first counter is incremented if a first test suite executed against the modified version of the operating platform fails. A second test suite can be executed against the modified version of the operating platform and test cases may be added to the first test suite based on whether the second test suite passes or fails.