Device Testing Relevance Scoring for Faster Error Debugging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current feature selection algorithms for debugging devices are inefficient as they ignore statistical significance, fail to capture geometric structure, and require manual analysis of numerous plots, making it tedious and time-consuming to determine variable dependencies and errors.
Innovation Solution
A tester and method that use relevance scores to evaluate subsets of input variables, providing plots indicating the impact of variable subsets on device errors, with scores calculated using entropy and believability measures to prioritize insightful plots and reduce debugging time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual analysis of all variable plots is performed to determine variable dependencies, then comprehensive error analysis can be achieved, but the time consumption and user effort become extremely high
Solution Approach 1:
The patent extracts only the most relevant variable subsets for error analysis by computing relevance scores that quantify the relationship between variable subsets and errors. Instead of requiring manual analysis of all possible plots, the system automatically identifies and extracts the top-k most relevant variable subsets, significantly reducing the number of plots users need to examine while maintaining comprehensive error analysis capability
Solution Approach 2:
The patent replaces the manual mechanical process of browsing and analyzing plots with an automated computational system. The evaluation unit automatically computes relevance scores using entropy-based measures and generates ranked plots, substituting human effort with algorithmic processing that can handle large numbers of variables and test cases efficiently
2Loss of information
If information theoretic measures are used for feature selection, then all information about errors can be captured, but geometric structure and statistical significance are ignored
Solution Approach 1:
The patent applies local quality by making the relevance score computation sensitive to local geometric properties of error distributions. The entropy-based measures evaluate the geometric structure of error patterns in variable space, allowing the system to identify variable subsets that not only contain information about errors but also present them in a geometrically structured way that is meaningful for debugging
Solution Approach 2:
The patent changes the parameters used for feature selection from traditional information theoretic measures alone to a composite approach that incorporates entropy-based relevance scores. This parameter change enables the system to simultaneously capture error information and recognize geometric structure by using entropy to measure both the information content and the geometric distribution of errors
3Device complexity
If feature selection algorithms are used to reduce the number of features, then the number of plots becomes more tractable, but statistical significance of data is ignored
Solution Approach 1:
The patent performs preliminary action by computing relevance scores and statistical significance measures before generating plots. The evaluation unit pre-calculates the relevance of each variable subset to errors using entropy-based measures and statistical tests, then uses these pre-computed scores to rank and filter plots. This preliminary analysis ensures that only statistically significant plots are presented, maintaining reliability while reducing complexity
Data Source
AI summary
A tester for testing a device under test is shown, having a test unit configured for performing a test of the device under test using multiple test cases, each test case having variable values of a set of predetermined variables, the test units configured to derive an output value for each test case indicating whether the device under test validly operates at a current test case or whether the device under test provides an error at the current test case; and an evaluation unit configured for evaluating the multiple test cases based on a plurality of subsets of the predetermined input variables with respect to the output value, the evaluation unit configured for providing a number of plots of the evaluation of the multiple test cases where each plot indicates the impact of one subset of the plurality of subsets of the predetermined input variables to the output value in dependence on respective relevance scores or associated with the respective relevance scores.