Software Callable-Unit Fuzzy Testing for Targeted Vulnerability Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity and size of vehicle software systems, combined with challenges of outsourcing, input and output fluctuation, and test-bed complexity, make security testing a time-consuming and incomplete process, leaving vehicles vulnerable to cyberattacks.
Innovation Solution
A grey-box fuzzy testing framework that prioritizes vulnerability exposure by using security vulnerability metrics to identify weak components and perform targeted testing, reducing complexity and ensuring thorough evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive security testing is performed on vehicle software systems, then vulnerability detection capability is improved, but testing time and resource consumption increase
Solution Approach 1:
The patent segments the software system into multiple callables and divides the testing process into discrete test cases, each targeting specific callables. This segmentation allows the testing system to focus resources on individual components rather than attempting to test the entire system monolithically, thereby improving vulnerability detection efficiency while managing testing time.
Solution Approach 2:
The patent performs preliminary actions by generating test cases in advance based on static analysis of the software system's call graph. Test cases are prepared and queued before execution, allowing the system to methodically work through predetermined test sequences. This preliminary preparation reduces runtime decision-making overhead and optimizes the testing process.
2Reliability
If comprehensive security testing is performed on vehicle software systems, then vulnerability detection capability is improved, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary testing system that acts as a mediator between the software system under test and the analysis processes. This testing system includes components like the test case generator, test executor, and call graph analyzer that intermediary between raw code and vulnerability detection, simplifying the overall process by providing structured intermediate representations (call graphs, test cases) that make the system more manageable.
Solution Approach 2:
The patent creates copies of the software system's structural information in the form of call graphs and test cases. Rather than directly analyzing the complex original system during testing, the system works with simplified copied representations that capture the essential relationships and control flow, reducing the complexity of the testing process while maintaining detection capability.
3Productivity
If targeted testing of vulnerable components is performed, then testing efficiency is improved, but measurement precision of vulnerability metrics is required
Solution Approach 1:
The patent replaces manual or heuristic-based vulnerability identification with automated static analysis mechanisms. The system automatically generates call graphs, identifies vulnerable callables through algorithmic analysis, and generates targeted test cases without relying on manual inspection. This substitution of mechanical/automated processes for manual methods improves both precision and efficiency.
Solution Approach 2:
The patent implements feedback loops where test results are analyzed and used to refine the testing process. The system monitors which callables are exercised by which test cases, identifies uncovered vulnerable components, and generates additional targeted test cases. This feedback mechanism continuously improves measurement precision of vulnerability metrics while maintaining high testing efficiency through automated iteration.
Data Source
AI summary
A method for a testing system to perform fuzzy testing of a software system, wherein the software system comprises a plurality of callable units and is arranged to receive input for the software system to process, the method comprising: determining, for each callable unit of the plurality of callable units, based on one or more security vulnerability metrics, a target number of times that callable unit is to be tested; initializing a ranked plurality of queues, each queue for storing one or more seeds, said initializing comprising storing one or more initial seeds in a corresponding queue of the ranked plurality of queues; performing a sequence of tests, wherein performing each test comprises: obtaining a seed from the highest ranked non-empty queue; performing a mutation process on the obtained seed to generate a test seed, wherein the mutation process is configured, at least in part, by mutation guidance information; providing the test seed as input to the software system for the software system to process; and evaluating the processing of the test seed by the software system to generate a result for the test; wherein each queue in the ranked plurality of queues has an associated seed addition criterion and wherein performing each test comprises either (a) adding the test seed to the highest ranked queue in the ranked plurality of queues for which the test seed meets the seed addition criterion associated with that queue; or (b) discarding the test seed if the test seed does not meet the seed addition criterion associated with any of the queues in the ranked plurality of queues; wherein the seed addition criteria are configured so that, if processing of a first test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest and if processing of a second test seed by the software system does not involve execution of, or an execution path approaching, a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added, wherein a callable unit is a callable unit of interest if the current number of tests that have resulted in execution of that callable unit is less than the target number of times that callable unit is to be tested.