ML-Predicted Coverage Feedback for Black Box Fuzzing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Black box fuzzing lacks feedback mechanisms to improve test case generation since it cannot access the source code of the target program, limiting its ability to optimize testing, especially in scenarios like embedded device software testing.

Innovation Solution

A method that uses a machine learning model to predict coverage information from program inputs, allowing for feedback-driven test case generation even without access to the target program's source code, effectively transforming black box fuzzing into a more effective gray box fuzzing approach.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If black box fuzzing is used to test target programs without source code access, then the testing can be applied to embedded devices and programs without deterministic outputs, but feedback mechanisms such as code coverage cannot be obtained to guide test case generation

Engineering Contradiction:
Improveapplicability to embedded devices and programs without deterministic outputsVSAvoidfeedback information such as code coverage
Core Design Contradiction:
Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent introduces an intermediary component (the machine learning model) that mediates between the black box fuzzer and the target program. This model learns to predict code coverage and other feedback information by analyzing the relationship between inputs and outputs, effectively creating a virtual feedback channel without requiring direct access to the target program's internal state or source code.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a copy or approximation of the target program's internal behavior through a trained machine learning model. This model replicates the coverage information that would otherwise be available only through direct access to the target program, allowing the fuzzer to operate with feedback as if it had gray box capabilities while maintaining black box operation.

Inventive Principle:
Principle #26Copying

2Ease of operation

If black box fuzzing operates without access to target program internals, then source code accessibility is maintained, but the fuzzer tests practically at random without ability to improve over time

Engineering Contradiction:
Improveno access required to target program internalsVSAvoidtest case generation efficiency
Core Design Contradiction:
Ease of operationVSProductivity

Solution Approach 1:

The patent implements a feedback mechanism where the machine learning model predicts coverage information based on input characteristics, and this predicted feedback is used to guide the generation of subsequent test cases. The fuzzer uses this feedback to prioritize inputs that are likely to explore new code paths, transforming random testing into a directed search that improves over time without requiring access to target program internals.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent performs preliminary training of the machine learning model using a dataset of inputs and their corresponding coverage information. This preliminary action equips the model with learned patterns that enable it to predict coverage for new inputs, allowing the fuzzer to make informed decisions from the start rather than testing purely at random.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20240354233A1Method for taking feedback into account in a software test
Publication Date: 2024.10.24 ROBERT BOSCH GMBH
  • US20240354233A1 patent drawing
  • US20240354233A1 patent drawing

AI summary

A method for taking feedback into account in a software test. The method includes: providing at least one target program to be tested; providing program inputs for executing at least one predetermined test case in the at least one target program by means of black box fuzzing; predicting coverage information on the basis of the provided program inputs, wherein the coverage information specifies an effect in the target program which results from the execution of the at least one predetermined test case; using the predicted coverage information as feedback for the software test.