Complexity-Based Sample Selection for Deep Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network training on large-scale datasets is inefficient due to high computing power and energy consumption, primarily because of the large number of parameters and the need for numerous labeled data samples.
Innovation Solution
An energy-efficient sample selection method based on complexity, employing inter-class sampling through reverse diverse self-paced learning and intra-class sampling using a density-based strategy to construct a lightweight dataset for model training, reducing the number of samples required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep neural network training is performed on large-scale datasets, then model performance is improved, but energy consumption increases and training efficiency decreases
Solution Approach 1:
The patent extracts and removes redundant or low-value samples from the large-scale dataset through two-stage sampling. First, inter-class sampling identifies and removes easy samples that do not contribute significantly to model performance. Second, intra-class sampling further filters samples within each class to retain only those with high information density. This extraction of essential samples reduces the dataset size while maintaining model performance, thereby reducing energy consumption during training.
Solution Approach 2:
The patent changes the parameter of sample selection criteria by introducing a two-stage sampling framework with different selection strategies for each stage. Inter-class sampling uses one criterion to select between classes, while intra-class sampling uses another criterion to select within classes. This parameter change in selection strategy enables more efficient sample selection that reduces training data volume without sacrificing model performance, thus lowering energy consumption.
2Reliability
If deep neural network training is performed on large-scale datasets, then model performance is improved, but training efficiency decreases
Solution Approach 1:
The patent extracts and removes redundant or low-value samples from the large-scale dataset through two-stage sampling. First, inter-class sampling identifies and removes easy samples that do not contribute significantly to model performance. Second, intra-class sampling further filters samples within each class to retain only those with high information density. This extraction of essential samples reduces the dataset size while maintaining model performance, thereby reducing energy consumption during training.
Solution Approach 2:
The patent changes the parameter of sample selection criteria by introducing a two-stage sampling framework with different selection strategies for each stage. Inter-class sampling uses one criterion to select between classes, while intra-class sampling uses another criterion to select within classes. This parameter change in selection strategy enables more efficient sample selection that reduces training data volume without sacrificing model performance, thus lowering energy consumption.
3Reliability
If the number of labeled data samples is increased, then model performance is improved, but computing power requirements increase
Solution Approach 1:
The patent extracts and removes redundant or low-value samples from the large-scale dataset through two-stage sampling. First, inter-class sampling identifies and removes easy samples that do not contribute significantly to model performance. Second, intra-class sampling further filters samples within each class to retain only those with high information density. This extraction of essential samples reduces the dataset size while maintaining model performance, thereby reducing energy consumption during training.
Solution Approach 2:
The patent changes the parameter of sample selection criteria by introducing a two-stage sampling framework with different selection strategies for each stage. Inter-class sampling uses one criterion to select between classes, while intra-class sampling uses another criterion to select within classes. This parameter change in selection strategy enables more efficient sample selection that reduces training data volume without sacrificing model performance, thus lowering energy consumption.
Data Source
AI summary
The invention discloses an energy-efficient sample selection method based on sample complexity, which performs sample selection on the raw data sets through two stages of inter-class sampling and intra-class sampling, the object is to select representative samples from large-scale data sets, thereby reducing the number of samples used for model training and achieving the object of lightweight training. Compared with the prior art, the invention has the following advantages: the invention proposes an energy-efficient sample selection method based on complexity, selects representative samples from large-scale datasets for efficient model training, and proves that sample complexity and model training strategies have a very important impact on the efficient training of deep neural networks. The invention also solves the problem of low efficiency of model training based on sample complexity and model training strategies, which has certain significance for alleviating the problem of low efficiency of deep learning model training.