Dithering Algorithm Selection for Deep Neural Network Parameter Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require a large number of arithmetic operations and generate many parameters during training, leading to increased size and computational complexity.
Innovation Solution
A method and system that utilize dithering algorithms to reduce data size by generating combinations of dithering operations on data groups, performing training operations on these reduced data sets, and selecting the combination resulting in the smallest steady deviation as a filter module to generate a data-recognition model with fewer parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deep neural network performs a large number of arithmetic operations during training, then the model accuracy is improved, but the size and computational complexity of the DNN increases
Solution Approach 1:
The patent extracts and removes redundant information from the training data through dithering operations. By applying dithering algorithms to quantize data to fewer bits, the patent reduces the amount of data that needs to be processed during training, thereby reducing DNN size and computational complexity while maintaining model accuracy through the preservation of essential data features
Solution Approach 2:
The patent changes the parameter representation by using dithering to quantize data to fewer bits. This parameter transformation reduces the data dimensionality and complexity, allowing the DNN to achieve the same accuracy with fewer computational operations and smaller model size
2Adaptability or versatility
If the DNN generates many parameters during training, then the model captures more features, but the storage requirements and processing load increase
Solution Approach 1:
The patent extracts only the essential features from the training data by applying dithering operations that remove redundant information. This selective extraction allows the DNN to learn and store fewer parameters while still capturing the most important features, thereby reducing storage requirements and processing load
Solution Approach 2:
The patent uses partial action by applying dithering to reduce data to fewer bits than the original precision. This partial quantization is sufficient to capture the essential features needed for model training, avoiding the need to store and process all original data details, thus reducing the quantity of parameters and storage requirements
Data Source
AI summary
A method for establishing a data-recognition model includes: generating (Z) number of Y-combinations of dithering algorithms from (X) number of dithering algorithms; for each Y-combination, performing a dithering operation on a to-be-processed data group, so as to obtain, in total, (Z) number of size-reduced data groups; performing training operations on a deep neural network using the size-reduced data groups, respectively, so as to generate, for each training operation, a DNN model and a steady deviation; and selecting the Y-combination corresponding to the size-reduced data group that results in the smallest steady deviation as a filter module, and selecting the corresponding DNN model as the data-recognition model.