Machine Learning Data Generation with Field Value Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data generation technologies for machine learning, such as VAE and GAN, face instability when the number of training data is small, and SMOTE only corrects inconsistencies in data format but not necessarily ensures appropriate values in structured data fields like network packet headers.
Innovation Solution
An information processing apparatus with a data generation unit and an adjustment unit that generates and adjusts input data for machine learning, using algorithms like SMOTE, VAE, or GAN, and an adjustment table to correct inconsistencies in structured data fields by replacing or recalculating values, ensuring consistency and appropriate values in fields like packet headers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If VAE or GAN is used for data generation, then the training data can be generated from small datasets, but the data generation becomes unstable when the number of training data is small
Solution Approach 1:
The patent segments the data generation process into two distinct stages: first generating candidate data using VAE/GAN, then filtering and correcting the generated data through grammar rule verification and field validation. This segmentation allows the system to leverage the data expansion capability of VAE/GAN while mitigating their instability through post-processing filters.
Solution Approach 2:
The patent implements feedback mechanisms where generated data is validated against grammar rules and field constraints, with invalid data being corrected or rejected. This feedback loop ensures that even when VAE/GAN produce unstable outputs from small training sets, the final training data meets quality standards through iterative validation and correction.
2Reliability
If SMOTE is used for data generation, then data can be stably generated even with small datasets, but the values in structured data fields may not be appropriate
Solution Approach 1:
The patent applies local quality by implementing field-specific validation and correction rules tailored to different data types. Each field in structured data (such as packet headers) has specific constraints and appropriate value ranges defined, allowing SMOTE-generated data to be corrected field-by-field to ensure local appropriateness while maintaining overall data stability.
Solution Approach 2:
The patent changes parameters by adjusting and correcting specific field values in generated data based on predefined constraints and appropriate ranges. When SMOTE generates data with inappropriate field values, the system modifies these parameter values to conform to expected formats and ranges, thereby improving manufacturing precision without sacrificing the stability advantage of SMOTE.
3Manufacturing precision
If only grammar rule correction is performed on generated data, then data format consistency is improved, but other inconsistencies in structured data fields remain
Solution Approach 1:
The patent implements a multi-functional correction system that performs multiple types of validation and correction in sequence: grammar rule verification for format consistency, field constraint validation for appropriate values, and cross-field dependency checking for logical consistency. This universal correction approach ensures comprehensive data quality improvement rather than addressing only a single aspect.
Solution Approach 2:
The patent applies continuous correction processes where generated data undergoes multiple successive validation and correction steps rather than a single pass. The correction action continues through grammar verification, field validation, and constraint checking until all inconsistencies are resolved, ensuring complete and thorough data correction rather than partial fixes.
Data Source
AI summary
An information processing apparatus includes a holding unit configured to hold first input data having a plurality of fields, for use in machine learning, a generation unit configured to generate second input data having a plurality of fields, for use in the machine learning, based on the first input data, and an adjustment unit configured to adjust values stored in the plurality of fields in the second input data.