Neural Network Parameter Masking for Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training artificial neural network (ANN) models for natural language processing (NLP) is computationally and memory-intensive, making it challenging for low-end computational systems and increasing the cost of implementation due to the need for high-end processors.
Innovation Solution
A method and system that optimize memory requirements by selecting and masking model parameters based on pre-defined rules, reducing the amount of memory needed for training, allowing ANN models to be trained on lower-end systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If high-end processors are used for training ANN models, then training capability and speed are improved, but implementation cost increases
Solution Approach 1:
The patent extracts and removes redundant model parameters from the training process by identifying and eliminating parameters that do not contribute to training accuracy. This reduction in parameter count directly decreases memory requirements, enabling training on lower-cost systems while maintaining training capability.
Solution Approach 2:
The patent changes the parameter representation by converting full-precision model parameters to reduced-precision formats. This parameter transformation maintains essential training functionality while significantly reducing memory consumption, thereby lowering implementation costs without sacrificing training capability.
2Manufacturing precision
If all model parameters are used for training, then model accuracy is improved, but memory consumption increases
Solution Approach 1:
The patent extracts only the essential model parameters required for accurate training by analyzing parameter importance and removing redundant parameters. This selective extraction maintains model accuracy while reducing the quantity of parameters that consume memory during training.
Solution Approach 2:
The patent applies partial action by using a subset of model parameters rather than all parameters. By identifying and utilizing only the critical parameters needed for training accuracy, the system reduces memory consumption while maintaining sufficient model accuracy for effective training.
3Ease of manufacture
If low-end computational systems are used, then implementation cost is reduced, but training capability deteriorates
Solution Approach 1:
The patent transforms model parameters to reduced-precision representations that require less memory to store and process. This parameter change enables low-end computational systems with limited memory resources to handle training tasks that would otherwise require high-end systems, maintaining training capability while reducing implementation cost.
Solution Approach 2:
The patent employs a simplified parameter representation approach that uses less computational resources. By adopting this lighter parameter format, low-end systems can perform training effectively without requiring expensive high-end hardware, making the training process accessible on budget-friendly computational platforms.
4Productivity
If memory allocation is increased for training, then training speed is improved, but system cost increases
Solution Approach 1:
The patent extracts and removes unnecessary model parameters from the training dataset. By eliminating redundant parameters, the total memory allocation required for training is reduced, allowing faster training speeds to be achieved on systems with limited memory resources without increasing system cost.
Data Source
AI summary
This disclosure relates to method and system for optimizing memory requirement for training an artificial neural network (ANN) model employed for natural language processing (NLP). In one embodiment, the method may include receiving a plurality of training parameters and a plurality of model parameters, selecting a set of model parameters from among the plurality of model parameters for training the ANN model based on a characteristic and an architecture of the ANN model, masking the set of model parameters in one or more layers of the ANN model based on a set of pre-defined rules to generate a set of masked model parameters, determining an amount of memory required for training the ANN model based on the set of masked model parameters, and providing the set of masked model parameters for training the ANN model when the amount of memory required is less than a determined threshold.