Neural Network Coverage Testing with Preferential Index Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing coverage test methods are ineffective for detecting bugs in programs created by machine learning, such as neural networks, as they often result in a coverage index of 100% for conditional branches, making it difficult to identify and detect bugs efficiently.
Innovation Solution
A coverage test support device and method that includes a storage unit for programs, a test case storage unit, a test input value generation unit, a measurement unit for calculating multiple coverage indexes, and a preferential coverage index selection unit, which identifies and prioritizes suitable coverage indexes to improve bug detection by analyzing neuron information and applying error backward propagation methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a coverage test is performed on a program created by machine learning using traditional coverage indexes, then the coverage index reaches 100%, but the ability to detect bugs is lost
Solution Approach 1:
The patent changes the measurement parameters from traditional conditional branch coverage to neuron activation coverage. By measuring which neurons are activated by test inputs and calculating coverage based on neuron activation patterns rather than conditional branches, the system achieves meaningful coverage metrics for neural network programs where traditional metrics fail to provide bug detection capability.
2Reliability
If multiple coverage indexes are calculated to improve bug detection, then the comprehensiveness of testing increases, but the complexity of the testing system increases
Solution Approach 1:
The patent segments the coverage measurement into distinct types (node coverage, edge coverage, neuron coverage) that can be calculated and evaluated separately. Each coverage type focuses on a specific aspect of the neural network's execution path, allowing comprehensive bug detection through multiple specialized metrics rather than one complex monolithic metric.
Solution Approach 2:
The patent creates a universal testing framework that can handle multiple coverage types through a unified approach. The system calculates various coverage indexes (node coverage, edge coverage, neuron coverage) using a common infrastructure that processes test inputs, traces execution paths, and aggregates results, making the system multi-functional without proportionally increasing complexity.
3Measurement precision
If test cases are generated to maximize neuron activation coverage, then the coverage index improves, but the number of test cases required increases
Solution Approach 1:
The patent implements feedback mechanisms where coverage results from previous test executions inform the generation of subsequent test cases. By analyzing which neurons remain unactivated or under-activated, the system generates new test inputs specifically designed to activate those neurons, creating a feedback loop that systematically improves coverage without requiring exhaustive random testing.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A coverage test support device includes a memory device that stores a test case and a specification content of each of a plurality of coverage indexes, and an arithmetic device that sequentially gives a test input value of each pair in the test case to a program created by a neural network, executes a predetermined number of tests, and acquires a test result of the tests and neuron information at the time of test execution, applies the acquired neuron information to the specification content of each of the plurality of coverage indexes and calculates a value for each coverage index, and identifies, among the coverage indexes, a coverage index in which an elongation rate of the calculated value shows a predetermined tendency, as a preferential coverage index that is to be used preferentially, when either the number of executions of the tests or the number of bugs in the test result exceeds a predetermined standard.