Data Records Selection for Efficient Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developers face challenges in ensuring data processing applications execute correctly in production environments without access to actual production data, as testing with large sets of records can be slow and impractical, and existing methods lack efficient mechanisms for selecting representative subsets of data records.
Innovation Solution
A computer-implemented method that accesses data records, analyzes their values, generates profiles, formulates subsetting rules based on these profiles, and selects subsets of data records to ensure complete code coverage and efficient testing, using techniques such as identifying target data fields, classifying values, and selecting records based on relationships and pseudofields.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large set of production data records is used for testing, then testing completeness and accuracy are improved, but testing time and computational resources increase significantly
Solution Approach 1:
The patent extracts only the essential and representative data records from the complete production data set based on profiling analysis. By identifying and selecting only those records that are necessary to exercise all application rules and logic paths, the system achieves comprehensive testing coverage with a much smaller subset of data, thereby reducing testing time while maintaining testing accuracy.
Solution Approach 2:
The patent performs data profiling and analysis before the actual testing process to pre-identify the minimal subset of data records that will provide complete code coverage. This preliminary action of analyzing data characteristics, relationships, and rule dependencies allows the system to prepare an optimized test data set in advance, avoiding the need to process large volumes of data during testing.
2Productivity
If data profiling and analysis are performed to identify representative subsets, then testing efficiency is improved, but data processing complexity increases
Solution Approach 1:
The patent segments the data processing task into distinct phases: data profiling, relationship identification, rule analysis, and subset selection. Each phase processes specific aspects of the data independently, breaking down the complex overall task into manageable components that can be executed systematically, thereby improving efficiency without overwhelming complexity.
Solution Approach 2:
The patent introduces an intermediary data profiling layer that acts as a mediator between the raw production data and the testing process. This intermediary layer analyzes and characterizes the data, identifying relationships and generating profiles that guide subsequent subset selection, thereby simplifying the complexity of directly processing raw data for testing purposes.
3Reliability
If all data fields and relationships are analyzed to ensure complete rule coverage, then application testing thoroughness is improved, but computational resources and processing time increase
Solution Approach 1:
The patent applies partial action by analyzing only the necessary data fields and relationships that are relevant to application rule execution. Rather than comprehensively analyzing all possible data characteristics, the system focuses specifically on identifying the minimal set of fields and relationships required to achieve complete rule coverage, thereby reducing computational resource consumption while maintaining thoroughness.
Data Source
AI summary
A computer-implemented method includes accessing a plurality of data records, each data record having a plurality of data fields. The method further includes analyzing values for one or more of the data fields for at least some of the plurality of data records and generating a profile of the plurality of data records based on the analyzing. The method further includes formulating at least one subsetting rule based on the profile; and selecting a subset of data records from the plurality of data records based on the at least one subsetting rule.