Neural Network Parameter Decomposition for Hardware Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems based on neural networks are resource-intensive due to the high number of arithmetic operations required, leading to significant energy consumption and performance challenges.
Innovation Solution
A method that optimizes the operation of a calculator implementing a neural network by decomposing neural network parameters into sub-values and assigning these sub-values to specific hardware blocks, optimizing performance metrics such as latency, energy consumption, and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of bits for processing data is increased to improve performance, then the performance of the neural network application is improved, but the energy consumption and computational resource requirements increase
Solution Approach 1:
The patent segments the neural network parameters into multiple sub-values with different precision levels. Instead of using uniform high-precision bits for all parameters, the system divides parameters into sub-values that can be processed with different bit depths depending on their contribution to the overall computation, thereby reducing total energy consumption while maintaining acceptable performance
Solution Approach 2:
The patent applies local quality by assigning different precision characteristics to different sub-values of parameters. Some sub-values receive higher precision treatment while others receive lower precision treatment, optimized according to their specific roles in the neural network computation. This localized optimization reduces overall resource requirements compared to uniform high-precision processing
2Measurement precision
If the number of bits for processing data is increased to improve performance, then the performance of the neural network application is improved, but the computational operator intensity and memory access requirements increase
Solution Approach 1:
The patent segments parameter values into sub-values that can be processed with different computational intensities. By dividing the parameter representation, the system can use simpler, less intensive computational operators for sub-values that contribute less to the overall computation, reducing the average operator intensity required across the entire neural network
Solution Approach 2:
The patent applies partial action by processing only the necessary portions of parameters with full precision. Instead of applying high-precision computation to all parameters uniformly, the system applies full precision only where needed (for sub-values with higher contribution) and uses reduced precision for less critical sub-values, thereby reducing overall computational operator intensity
3Measurement precision
If the number of bits for processing data is increased to improve performance, then the performance of the neural network application is improved, but the memory access requirements increase
Solution Approach 1:
The patent segments parameter storage into multiple sub-value representations with different memory requirements. By dividing parameters into sub-values, the system can store and access only the necessary precision levels for each sub-value, reducing total memory usage compared to storing all parameters at full precision
Solution Approach 2:
The patent applies local quality to memory storage by assigning different storage requirements to different sub-values based on their computational importance. Critical sub-values are stored with higher precision while less critical sub-values use reduced precision storage, optimizing the balance between performance and memory consumption
Data Source
AI summary
A method is provided for optimizing the operation of a calculator implementing a neural network, the method comprising providing a neural network, providing training data relating to the values taken by the neural network parameters during a training of the neural network on a test database, determining, depending on the training data, an implementation of the neural network on hardware blocks of a calculator so as to optimize a cost relating to the operation of said the calculator implementing the neural network, the implementation being determined by decomposing the values of the neural network parameters into sub-values and by assigning to every sub-value, one hardware block from a set of hardware blocks of the calculator, and the operation of the calculator with the determined implementation.