Neural Network Input Layer Sparsification for Training Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art neural networks face inefficiencies in training due to the exponential increase in time and storage requirements with large datasets, and the quality of training data significantly impacts accuracy, making it impractical to analyze and improve the quality of individual datapoints.
Innovation Solution
Sparsification of neural network layers, particularly the input layer, is implemented to eliminate nodes with insignificant or adverse weights, filtering out poor-quality training data and reducing the dataset size, thereby improving training efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the size of training dataset is increased to improve neural network accuracy, then accuracy is improved, but training time increases exponentially
Solution Approach 1:
The patent extracts and eliminates nodes from the neural network input layer that correspond to neutral or adverse training data, separating useful information from harmful information. This extraction process filters out problematic data points that would otherwise slow down training, allowing the network to train faster on a reduced, high-quality dataset.
Solution Approach 2:
The patent changes the parameter of data quality by transforming the training dataset from a potentially noisy large dataset to a filtered high-quality dataset. By modifying which data points are included in training based on node importance analysis, the system achieves both reduced training time and maintained or improved accuracy.
2Measurement precision
If the size of training dataset is increased to improve neural network accuracy, then accuracy is improved, but storage requirements increase
Solution Approach 1:
The patent extracts and removes nodes corresponding to neutral or adverse training data from the input layer, thereby extracting and eliminating redundant or harmful data from storage. This reduces the quantity of training data that needs to be stored while maintaining the quality and effectiveness of the remaining training dataset.
Solution Approach 2:
The patent discards neutral or adverse training data that provides no value or harmful effects to the neural network training. By identifying and discarding these problematic data points, the system recovers storage space while improving the overall quality of the training data that remains.
3Stability of the object's composition
If all training data is used to ensure comprehensive coverage, then data completeness is maintained, but training efficiency decreases
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different data points based on their individual impact on training. Instead of uniformly treating all data, the system identifies and selectively includes only those data points that have positive effects on training accuracy, while excluding neutral or adverse data points.
Solution Approach 2:
The patent extracts and removes nodes from the input layer that correspond to neutral or adverse training data, thereby separating useful information from harmful information. This extraction process filters out problematic data points that would otherwise slow down training, allowing the network to train faster on a reduced, high-quality dataset.
4Productivity
If nodes with insignificant weights are eliminated to reduce network complexity, then training speed is improved, but network accuracy may be compromised
Solution Approach 1:
The patent uses feedback from training accuracy analysis to guide the elimination process. By continuously monitoring training performance and using this feedback to identify which node eliminations are safe, the system ensures that accuracy is maintained while achieving speed improvements through the removal of insignificant nodes.
Solution Approach 2:
The patent changes the parameter of network structure by transforming from a dense input layer to a sparsified input layer with fewer nodes. This structural change reduces computational complexity and training time while maintaining accuracy through careful selection of which nodes to eliminate based on their weight significance.
Data Source
AI summary
A device, system, method and non-transitory computer-readable storage medium for filtering a dataset for training a neural network. Initial instances of recorded input samples of source recording device(s) may be encoded, according to an input map, from the training dataset to respective nodes in the neural network's input layer. The input layer of the neural network may be sparsified by eliminating its nodes during a training phase. The training dataset may be filtered to exclude subsequent instances of recorded input samples from the source recording devices encoded in the eliminated nodes. Subsequent instances of the recorded input samples of the filtered training dataset may be encoded to remaining nodes not eliminated (and not to eliminated nodes) in the input layer to train the neural network in a subsequent training phase or generate a prediction output of the neural network in a prediction phase.