Software Testing System for Discrimination Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software testing methods face challenges in efficiently generating and analyzing test cases for complex software systems, particularly in identifying and measuring discrimination across various input characteristics, due to high complexity and adaptive rules, which can lead to incomplete or misleading results.
Innovation Solution
A system and method for efficient software testing that identifies input characteristic subsets, generates discrimination test cases, and performs pruning to reduce the number of test cases, while determining discrimination scores through causal experimentation, using output domain distance functions and discrimination scores to measure fairness and discrimination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a brute force approach is used to test every possible input combination, then completeness of testing is improved, but the number of test cases becomes massive and difficult to analyze
Solution Approach 1:
The patent segments the input space by identifying characteristic subsets that define discrimination scenarios. Instead of testing all possible inputs, the system divides the input space into meaningful segments based on characteristic combinations, allowing focused testing on discriminative patterns while reducing overall test case volume.
Solution Approach 2:
The patent extracts and focuses on discriminative patterns by identifying characteristic subsets that are most likely to reveal discrimination. The pruning operation extracts only the essential test cases that expose discrimination, removing redundant test cases from the massive set while preserving testing effectiveness.
2Productivity
If the number of test cases is reduced through pruning, then analysis efficiency is improved, but risk of missing discrimination patterns increases
Solution Approach 1:
The patent performs preliminary identification of characteristic subsets and generates discrimination test cases focused on potential discrimination patterns before pruning. This preliminary action ensures that only relevant test cases are considered for pruning, maintaining detection accuracy while enabling efficient analysis of the reduced test set.
Solution Approach 2:
The system uses discrimination scores as feedback to guide the pruning process. Test cases are evaluated based on their ability to detect discrimination, and the pruning operation retains test cases with high discrimination scores while removing low-value cases. This feedback mechanism ensures that pruning does not compromise detection accuracy.
Data Source
AI summary
A system includes a memory system having computer readable instructions and one or more processors for executing the computer readable instructions. The computer readable instructions include identifying input characteristic subsets for a system under test based on an input schema, generating discrimination test cases including a combination of inputs for the system under test based on the input characteristic subsets, and performing a pruning operation to reduce a number of the discrimination test cases based on identifying a superset of input characteristics that includes at least one of the input characteristic subsets having a discrimination score above a threshold value. The computer readable instructions further include determining discrimination scores based on applying one or more of the discrimination test cases remaining after the pruning operation to the system under test and observing an output, and outputting a measurement of discrimination of the system under test based on the discrimination scores.