Machine Learning Model Parameter Reduction for Inference Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High computational complexity in machine learning models hinders efficient inference operations due to high inference accuracy requirements, leading to increased power consumption and reduced processing speed, while reducing weight parameters to lower complexity often compromises accuracy.
Innovation Solution
An arithmetic operation device that generates a second model by removing intermediate layers and adjusting weight parameters from a first model, allowing for reduced computational complexity while maintaining desired inference accuracy through a training method that determines the optimal number of parameters and layers based on required performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of parameters and intermediate layers is increased to improve inference accuracy, then accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies dynamics by making the model configuration adjustable rather than fixed. The arithmetic operation device can dynamically select different models (first model with full parameters, second model with reduced parameters) based on the specific inference requirements, allowing the system to adapt between accuracy and complexity depending on the task demands.
Solution Approach 2:
The patent changes the parameter configuration by creating multiple versions of the machine learning model with different numbers of parameters and intermediate layers. The system stores both a first model with full parameters and a second model with reduced parameters, enabling parameter variation to balance between inference accuracy and computational complexity.
2Productivity
If computational complexity is reduced by removing parameters, then power consumption decreases and processing speed increases, but inference accuracy deteriorates
Solution Approach 1:
The system dynamically selects which model to use based on the inference task requirements. When high processing speed is needed, the arithmetic operation device selects the second model with reduced parameters; when high accuracy is required, it selects the first model with full parameters. This dynamic adaptation resolves the contradiction between speed and accuracy.
Solution Approach 2:
The patent applies partial action by removing only certain intermediate layers and parameters to create the second model, rather than completely redesigning the model architecture. This partial reduction achieves lower computational complexity while maintaining sufficient accuracy for many practical applications.
3Device complexity
If a single model is used for all inference tasks, then device complexity is reduced, but the model cannot be optimized for specific performance requirements
Solution Approach 1:
The patent segments the model into multiple versions (first model with full parameters, second model with reduced parameters) that can be stored and selected as needed. This segmentation allows each model to be optimized for specific requirements while keeping the overall system management simple through a unified storage and selection mechanism.
Solution Approach 2:
The arithmetic operation device is designed with multi-functionality to handle different inference requirements by selecting from multiple pre-trained models. The device can perform both high-accuracy inference using the first model and high-speed inference using the second model, making it universally applicable to various performance requirements without increasing structural complexity.
Data Source
AI summary
An arithmetic operation device is provided that removes a part of parameters of a predetermined number of parameters from a first machine learning model which includes the predetermined number of parameters and is trained so as to output second data corresponding to input first data, determines the number of bits of a weight parameter according to required performance related to an inference to generate a second machine learning model, and acquires data output from the second machine learning model so as to correspond to the input first data with a smaller computational complexity than the first machine learning model.