ANN-Based Data Sampling for Representative Stratified Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data sampling methods, such as random and stratified sampling, often suffer from selection bias and fail to ensure a representative sample, especially when dealing with populations having multiple categories and varying attribute distributions.
Innovation Solution
A method and system utilizing an artificial neural network (ANN) model to identify numerical and categorical data columns, determine predictor and predictand variables through linear regression, and perform stratified sampling to generate a sample key, ensuring a representative sample dataset is created.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If random sampling technique is used, then sampling process is simple and fast, but sampling bias occurs and sample representativeness deteriorates
Solution Approach 1:
The patent introduces an intermediary classification system that mediates between random sampling and representative sampling. By classifying population data into strata based on key attributes before sampling, the system ensures that samples are drawn from each stratum proportionally, eliminating sampling bias while maintaining the simplicity of random selection within each stratum.
Solution Approach 2:
The patent segments the population data into multiple strata based on categorical attributes and key numerical attributes. This segmentation allows the sampling process to address different segments separately, ensuring that each segment is adequately represented in the final sample, thereby improving sample representativeness without significantly increasing complexity.
2Reliability
If stratified sampling technique is used, then sampling bias is reduced, but device complexity and processing time increase
Solution Approach 1:
The patent performs preliminary classification of population data into strata based on key attributes before the actual sampling process. This preliminary action organizes the data in advance, making the subsequent sampling process simpler and more efficient. The classification is done once and reused, reducing the overall complexity of the sampling process.
Solution Approach 2:
The patent changes the parameters used for stratification by identifying key numerical attributes and categorical attributes that best represent the population. By optimizing these parameters, the system achieves effective stratification with fewer strata, reducing processing complexity while maintaining sample representativeness.
3Reliability
If conventional stratified sampling is used with single category, then sampling bias is partially reduced, but sample accuracy deteriorates for populations with multiple categories
Solution Approach 1:
The patent segments the population into multiple strata based on combinations of categorical attributes and key numerical attributes, rather than using a single category. This multi-dimensional segmentation ensures that samples accurately represent the complex structure of populations with multiple categories, improving sample accuracy while maintaining bias reduction.
Solution Approach 2:
The patent creates a universal sampling framework that can handle populations with any number and combination of categories. The system identifies key attributes dynamically and applies stratification based on these attributes, making the sampling method universally applicable to diverse population structures while maintaining high sample accuracy.
Data Source
AI summary
This disclosure relates to method and system for data sampling using an artificial neural network (ANN) model. In an embodiment, the method includes identifying a plurality of numerical data columns and a plurality of categorical data columns in population dataset, determining a set of predictor variables and a set of predictand variables by applying a linear regression on the plurality of numerical data columns, generating a sequential prediction model based on the set of predictor variables and the set of predictand variables, and performing stratified sampling on the plurality of categorical data columns to generate a set of stratified samples. The method further includes generating a sample key based on the set of stratified samples and the sequential prediction model, and generating a sample dataset representative of the population dataset based on the sample key.