Test Data Generation via Stratified Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in constructing a data platform is the limited availability of actual data for testing, leading to potential skew in training data when using randomly sampled test data, which can affect the quality and performance of the data platform.
Innovation Solution
A method is implemented to classify medical records into groups based on execution characteristics and generate test data that preserves the proportions of these records, ensuring that the test data accurately represents the original data, thereby improving the evaluation of the data platform.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If randomly sampled test data is used, then test data can be generated quickly, but the training data becomes skewed and does not represent actual data accurately
Solution Approach 1:
The patent segments the data selection process into multiple stages: first dividing actual data into training set and validation set, then further segmenting the validation set into stratified groups based on execution characteristics. This segmentation allows random sampling to occur within each stratum rather than across the entire dataset, ensuring both speed and representativeness are maintained.
Solution Approach 2:
The patent applies local quality by ensuring that each stratified group within the validation set maintains the same execution characteristic distribution as the original actual data. This means different parts of the test data (different strata) have specialized quality properties that collectively preserve the overall data distribution, preventing skew while enabling efficient random sampling.
2Manufacturing precision
If a large amount of actual data is used for testing, then test data quality improves, but data availability is limited in certain scenarios
Solution Approach 1:
The patent performs preliminary action by pre-dividing the limited actual data into training and validation sets before testing begins. The validation set is then pre-stratified into groups based on execution characteristics, so that when test data is needed, the system can efficiently sample from these pre-organized strata without requiring additional data collection or processing during the testing phase.
Solution Approach 2:
The patent creates a copied representation of the actual data distribution through stratified sampling. Instead of needing the full actual data dataset during testing, the system creates a test validation set that copies the essential execution characteristic distribution of the original data, preserving data quality while working within quantity constraints.
3Measurement precision
If test data is stratified by execution characteristics, then data platform evaluation accuracy improves, but data processing complexity increases
Solution Approach 1:
The patent segments the validation set into stratified groups based on execution characteristics, where each segment represents a specific type of data processing scenario. This segmentation enables accurate evaluation by ensuring all relevant execution paths are represented in the test data, while the segmentation itself is performed once during data preparation rather than during each evaluation run.
Solution Approach 2:
The patent changes the parameter of data organization from unstructured or简单地随机分布 to structured stratification based on execution characteristics. This parameter change improves measurement precision by ensuring representative sampling across different data types, while the stratification structure is established upfront and can be reused across multiple evaluation runs, limiting the increase in processing complexity.
Data Source
AI summary
Test data may be generated by classifying a plurality of medical records of patients into a plurality of groups, based on at least an execution characteristic of a processing program that processes the medical records, and generating test data including at least a part of the plurality of medical records based on proportions of the number of the medical records classified into the plurality of groups.