Image Model Quantization With Correction Layers for Faster Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing deep learning frameworks like Caffe support only single-precision and double-precision floating-point data types, leading to time-consuming and memory-intensive processes, especially in embedded devices, making it challenging to balance data processing speed and accuracy for large networks.
Innovation Solution
An image processing method that converts the data type of initial models to int8 or half-precision, adds a correction layer, and optimizes weights to achieve an image processing model with improved speed and accuracy, using a deep learning framework like TensorFlow or Caffe.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If float data type is used for deep learning model processing, then data processing accuracy is maintained, but data processing speed decreases and memory consumption increases
Solution Approach 1:
The patent changes the data type parameter from float to int8 or half-precision formats, which reduces the bit width and memory footprint while maintaining acceptable accuracy through quantization techniques and correction mechanisms
Solution Approach 2:
The system dynamically selects between different data types (float, half-precision, int8) based on the specific processing task requirements, allowing flexibility to optimize between speed and accuracy for different scenarios
2Measurement precision
If float data type is used for deep learning model processing, then data processing accuracy is maintained, but memory consumption increases
Solution Approach 1:
The patent changes the data type parameter from float to int8 or half-precision formats, which reduces the bit width and memory footprint while maintaining acceptable accuracy through quantization techniques and correction mechanisms
Solution Approach 2:
The patent introduces correction layers that use lower-precision data types for the majority of computations, accepting small accuracy losses that are then corrected by additional processing, effectively using 'cheaper' computational resources for most operations
3Productivity
If data type is converted to int8 or half-precision, then data processing speed increases, but data processing accuracy may decrease
Solution Approach 1:
The patent introduces correction layers as intermediary components that compensate for the accuracy loss introduced by low-precision data types, acting as a mediator between the speed benefits of int8/half-precision and the accuracy requirements of the task
Solution Approach 2:
The system uses correction layers that provide feedback mechanisms to adjust and compensate for quantization errors, continuously refining the output to maintain accuracy while benefiting from faster processing
Data Source
AI summary
An image processing method and an electronic device are disclosed, the method acquires training data in response to receiving an image processing instruction and trains a deep learning model with the training data by using a preset deep learning framework to obtain an initial model. A data type of the initial model is converted to increase the data processing speed of the model. A correction layer is added to the converted initial model and the initial model is optimized by training weights of the correction layer to obtain an image processing model to further increase the data processing speed. After acquiring an image from the image processing instruction, the image processing model can be used to process the image and an image processing result can be outputted. The image is processed based on the optimized image processing model, and the data processing speed and accuracy are guaranteed.