Multi-Variable Data Generation with Binning and Contingency Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data generation processes struggle with complex data sets containing multiple variable types, often focusing on single variables or small sets, leading to inefficiencies in generating accurate new data.
Innovation Solution
A system that splits original data sets into continuous and discrete subsets, converts continuous variables to discrete using dimension reduction and binning, and generates new data through contingency tables, creating a new data set with similar characteristics to the original.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data generation is performed on complex multi-variable data sets, then the quality and accuracy of generated training data is improved, but the computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the complex multi-variable data set into multiple subsets based on variable types (continuous, discrete, categorical). Each subset is processed independently through type-specific generation methods, reducing the overall computational complexity while maintaining accuracy. The segmented subsets are then combined to form the complete generated data set.
Solution Approach 2:
The patent transforms the complex multi-variable problem into multiple simpler sub-problems by adding a dimension of classification based on variable types. This allows the system to apply different processing strategies to different variable categories, making the overall process more manageable and efficient.
2Ease of manufacture
If traditional data generation methods are used on complex data sets, then the process is simpler, but the accuracy and quality of generated data deteriorates
Solution Approach 1:
The patent applies different data generation methods tailored to specific variable types within the data set. Continuous variables receive continuous distribution-based generation, discrete variables receive discrete distribution-based generation, and categorical variables receive contingency table-based generation. This localized approach ensures each variable type is handled with the most appropriate method, improving overall data quality.
Solution Approach 2:
The patent changes the parameters and methods of data generation based on the type of variables present in the data set. By detecting variable types and adjusting the generation approach accordingly, the system maintains simplicity in implementation while achieving high accuracy in the generated data.
3Speed
If single variable or small set data generation is performed, then the processing speed is faster, but the usefulness for complex machine learning models is reduced
Solution Approach 1:
The patent segments the complex data generation task into smaller sub-tasks for different variable types, allowing parallel processing that maintains speed while handling complexity. Each segment is processed independently and quickly, then combined to form the complete data set suitable for complex machine learning models.
Solution Approach 2:
The patent creates a universal data generation system that can handle multiple variable types (continuous, discrete, categorical) and generate comprehensive multi-variable data sets. This multi-functional approach makes the generated data applicable to a wide range of complex machine learning models while maintaining efficient processing.
Data Source
AI summary
An example operation may include one or more of storing an original data set in memory, splitting the original data set into a subset of continuous-type data values and a subset of discrete-type data values based on variable types in the original data set, converting the subset of continuous-type data values into a second subset of discrete-type data values based on a data binning operation, generating a new subset of continuous-type data values based on the subset of continuous-type data values in the original data set, and combining a subset of discrete-type data values from a conditional contingency table within the new subset of continuous-type data values to generate a new data set.