Machine Learning Data Sample Selection via Feature Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training a machine learning model with a large dataset can increase computational load and extend training time, and the selection of data samples affects prediction accuracy, with existing methods failing to efficiently narrow down relevant samples for improved accuracy.
Innovation Solution
A method involving an information processing apparatus that extracts features from input data, normalizes them, and selects a subset of data samples based on these normalized features using principal component analysis to reduce the dataset size and improve prediction accuracy, specifically by using a graph neural network to update node features based on neighboring nodes and selecting samples that are dissimilar to others.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large dataset is used to train the machine learning model, then prediction accuracy is improved, but computational load and training time increase
Solution Approach 1:
The patent extracts and selects only the most relevant data samples from the large dataset for training. By using feature extraction and normalization followed by selective sampling, the system identifies and trains on the subset of data that provides the highest prediction accuracy, thereby reducing training time while maintaining or improving accuracy compared to using all data
Solution Approach 2:
The patent applies parameter changes through feature normalization, transforming the data into a standardized format where features have zero mean and unit variance. This parameter transformation enables more efficient training by improving convergence speed and allowing the use of smaller, normalized datasets without sacrificing prediction accuracy
2Measurement precision
If all data samples are used for training, then comprehensive information is captured, but computational resources are wasted on irrelevant samples
Solution Approach 1:
The system extracts only the essential information needed for training by selecting a representative subset of data samples. Through feature extraction and normalization, it identifies the most informative samples that capture the key patterns, eliminating redundant computational processing on irrelevant or duplicate samples
Solution Approach 2:
The patent segments the large dataset into smaller, manageable subsets based on feature similarity and representativeness. By dividing the data into meaningful groups and selecting samples from each segment, the system reduces computational load while ensuring comprehensive coverage of the data distribution
Data Source
AI summary
An information processing apparatus enters a plurality of data samples individually to a machine learning model and extracts a plurality of features from the machine learning model. The information processing apparatus normalizes the plurality of features to a plurality of normalized features. The information processing apparatus selects, based on the plurality of normalized features, at least one data sample, which is part of the plurality of data samples, from the plurality of data samples. The information processing apparatus trains the machine learning model by using the at least one data sample.