Machine Learning Model Simplification via Weight Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep machine learning models consume substantial resources such as storage, memory bandwidth, and energy during training and inference, making it difficult to deploy them on mobile and embedded devices.
Innovation Solution
A method that involves training a machine learning model by evaluating and removing connections with weights that do not satisfy a threshold condition, and generating a filter to process input data, thereby reducing the model's complexity and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep machine learning models are used to improve model accuracy, then model accuracy is improved, but storage, memory bandwidth, energy consumption, and computational resources increase substantially
Solution Approach 1:
The patent extracts and removes unnecessary connections from the machine learning model based on weight evaluation. By identifying and eliminating connections with weights below a threshold, the model structure is simplified while preserving essential functionality, thereby reducing energy consumption without significantly compromising accuracy
Solution Approach 2:
The patent changes the parameter of connection weights by evaluating them and removing those that do not satisfy threshold conditions. This parameter-based filtering approach transforms the dense model into a sparse model, reducing computational complexity and energy requirements while maintaining predictive performance
2Measurement precision
If deep machine learning models are used to improve model accuracy, then model accuracy is improved, but storage and memory bandwidth requirements increase substantially
Solution Approach 1:
The patent extracts unnecessary connections from the model based on weight evaluation. By removing connections with insufficient weights, the storage requirements for model parameters are reduced while retaining the essential knowledge needed for accurate predictions
Solution Approach 2:
The patent applies parameter-based filtering to identify and remove connections with weights below a threshold. This transforms the model from dense to sparse representation, significantly reducing storage requirements and memory bandwidth needs while preserving model accuracy
3Measurement precision
If deep machine learning models are used to improve model accuracy, then model accuracy is improved, but device complexity increases making deployment difficult
Solution Approach 1:
The patent extracts and removes unnecessary connections from the model structure. This simplification reduces the complexity of deploying the model on mobile and embedded devices while maintaining the accuracy provided by the deep learning architecture
Solution Approach 2:
The patent uses parameter thresholding to filter connections, transforming the complex dense model into a simpler sparse model. This parameter-based simplification makes the model more suitable for deployment on resource-constrained devices while preserving predictive performance
Data Source
AI summary
Embodiments of the disclosure provide methods and systems for performing machine learning. The method can include: receiving training data; training a machine learning model based on the training data, wherein the machine learning model includes multiple layers each having one or more nodes having one or more connections with a node from another layer of the machine learning model; evaluating weights associated with the connections of the machine learning model, wherein each connection has a corresponding weight; removing, from the machine learning model, one or more connections having a weight that does not satisfy a threshold condition; and after the connections have been removed, updating the machine learning model.