ML-Predicted Coverage Feedback for Black Box Fuzzing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Black box fuzzing lacks feedback mechanisms to improve test case generation since it cannot access the source code of the target program, limiting its ability to optimize testing, especially in scenarios like embedded device software testing.
Innovation Solution
A method that uses a machine learning model to predict coverage information from program inputs, allowing for feedback-driven test case generation even without access to the target program's source code, effectively transforming black box fuzzing into a more effective gray box fuzzing approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If black box fuzzing is used to test target programs without source code access, then the testing can be applied to embedded devices and programs without deterministic outputs, but feedback mechanisms such as code coverage cannot be obtained to guide test case generation
Solution Approach 1:
The patent introduces an intermediary component (the machine learning model) that mediates between the black box fuzzer and the target program. This model learns to predict code coverage and other feedback information by analyzing the relationship between inputs and outputs, effectively creating a virtual feedback channel without requiring direct access to the target program's internal state or source code.
Solution Approach 2:
The patent creates a copy or approximation of the target program's internal behavior through a trained machine learning model. This model replicates the coverage information that would otherwise be available only through direct access to the target program, allowing the fuzzer to operate with feedback as if it had gray box capabilities while maintaining black box operation.
2Ease of operation
If black box fuzzing operates without access to target program internals, then source code accessibility is maintained, but the fuzzer tests practically at random without ability to improve over time
Solution Approach 1:
The patent implements a feedback mechanism where the machine learning model predicts coverage information based on input characteristics, and this predicted feedback is used to guide the generation of subsequent test cases. The fuzzer uses this feedback to prioritize inputs that are likely to explore new code paths, transforming random testing into a directed search that improves over time without requiring access to target program internals.
Solution Approach 2:
The patent performs preliminary training of the machine learning model using a dataset of inputs and their corresponding coverage information. This preliminary action equips the model with learned patterns that enable it to predict coverage for new inputs, allowing the fuzzer to make informed decisions from the start rather than testing purely at random.
Data Source
AI summary
A method for taking feedback into account in a software test. The method includes: providing at least one target program to be tested; providing program inputs for executing at least one predetermined test case in the at least one target program by means of black box fuzzing; predicting coverage information on the basis of the provided program inputs, wherein the coverage information specifies an effect in the target program which results from the execution of the at least one predetermined test case; using the predicted coverage information as feedback for the software test.

