Code Coverage Sampling for Tax Software Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data management systems face inefficiencies and inaccuracies in testing new processes due to resource-intensive methods, often failing to represent rare user attributes, leading to delays and inaccuracies in updating tax return preparation systems.
Innovation Solution
The system generates efficient sample test sets by analyzing historical user data, executing prior code, and creating code coverage representation values to group users based on executed code sections, allowing for a small sample size to represent a wide range of user attributes, reducing resource usage and ensuring accurate testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a very large number of historical users are used for testing, then testing coverage is improved, but computing and human resources are excessively consumed
Solution Approach 1:
The patent segments the historical user population into distinct groups based on code coverage representation values. Users are clustered into groups where each group represents a unique combination of code execution patterns. This segmentation allows testing to focus on representative samples from each group rather than processing all individual users, thereby reducing computing resources while maintaining comprehensive testing coverage across all code paths.
Solution Approach 2:
The patent performs preliminary analysis by executing the prior version of software code against all historical users beforehand to generate code coverage representation values and cluster users into groups. This pre-processing step creates a structured framework that enables efficient sampling during actual testing. The preliminary action of clustering users based on code coverage patterns allows the system to select minimal representative samples that guarantee coverage of all code paths without re-processing entire user datasets during testing.
2Quantity of substance
If a smaller random sample of historical users is used for testing, then resource usage is reduced, but uncommon user attributes are likely to be missed
Solution Approach 1:
The patent transforms the sampling approach by changing the parameter used for selection from random sampling to sampling based on code coverage representation values. Instead of selecting users randomly, the system uses predetermined clusters based on code execution patterns as the selection criterion. This parameter change ensures that every cluster representing a unique code combination is represented in the test sample, guaranteeing coverage of uncommon user attributes while using a small sample size that conserves computing resources.
3Reliability
If extensive testing is performed to ensure accuracy, then reliability is improved, but time and expense increase
Solution Approach 1:
The patent performs preliminary execution of the prior software version against all historical users to generate code coverage representation values and create user clusters before actual testing begins. This pre-computed clustering structure enables rapid selection of representative test samples that guarantee coverage of all code paths. During actual testing, the system only needs to process small samples from each pre-defined cluster, dramatically reducing testing time while maintaining comprehensive coverage and high reliability through the guaranteed representation of all code execution scenarios.
Data Source
AI summary
A method and system generates sufficient sample data sets for efficiently and accurately testing a new calculation for preparing a portion of an electronic document for users of an electronic document preparation system. The method and system prepares the new calculation and gathers historical user data related to previously prepared electronic documents for a large number of historical users. The method and system generates a representative value for each historical user data, based on the sections of a previous version of electronic document preparation software which were executed for each historical user in preparing electronic documents for the historical users. The method and system groups the historical users based on the hash values which indicates the historical user's behavior in the software. The groups are then sampled by selecting a small number of historical users from each group.