Grouped Parameter Quantization for Edge Deep Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models are becoming increasingly complex, leading to high resource consumption due to the large number of parameters and calculations, which poses challenges in storage and data transmission, especially for edge devices with limited resources.
Innovation Solution
The method involves dividing model parameters into groups and adjusting their data type from a first data type to a second data type, such as converting floating-point to integer type, to reduce storage and transmission requirements while maintaining model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models use more complex network structures with more parameters, then model accuracy and capability are improved, but storage space and resource consumption increase
Solution Approach 1:
The patent applies parameter changes by converting model parameters from floating-point data type to integer data type. This changes the data representation parameters to reduce storage requirements while maintaining computational accuracy. The conversion involves dividing parameters into groups and adjusting their data types, which directly addresses the storage space issue without sacrificing model performance.
2Measurement precision
If deep learning models have more parameters and higher calculation complexity, then model capability is improved, but data transmission requirements increase
Solution Approach 1:
The patent changes the data type parameter of model parameters from floating-point to integer representation. This parameter change reduces the amount of data that needs to be transmitted during model deployment and inference, especially important for edge devices with limited bandwidth. The grouped parameter conversion ensures minimal loss of precision during the data type transformation.
3Measurement precision
If floating-point data type is used for model parameters, then calculation precision is maintained, but storage and transmission efficiency deteriorate
Solution Approach 1:
The patent transforms the data type parameter from floating-point to integer for model parameters. This changes the representation efficiency while maintaining calculation precision through careful grouping and scaling of parameters. The integer representation achieves better storage and transmission efficiency without significantly compromising the computational accuracy needed for model inference.
Data Source
AI summary
A method of processing data for a target model, an electronic device, and a storage medium, which relate to a field of deep learning. The method of processing data for a target model includes: acquiring the target model, wherein the target model includes at least one network layer, and each network layer of the at least one network layer includes a plurality of model parameters; for a target network layer in the at least one network layer, dividing the plurality of model parameters of the target network layer into a plurality of groups; and adjusting a data type of the model parameters in the plurality of groups from a first data type to a second data type respectively, so as to obtain a processed target model.