Neural Network Quantization With Privacy-Preserving Feature Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face challenges in quantization that reduce operation speed and memory usage while compromising accuracy, and there is a difficulty in reflecting user data privacy and personalization due to the need for training data collection.
Innovation Solution
An electronic apparatus quantizes a neural network model, transmits information to an external device, receives feature and identification information, and trains the model based on this data to update parameters, ensuring privacy and personalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If quantization is performed to reduce bit number of integers, then operation speed and memory usage are improved, but accuracy is greatly reduced
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the bit width of integer representations based on the layer type and importance within the neural network. Critical layers maintain higher precision with more bits, while less critical layers use lower precision with fewer bits, optimizing the balance between operation speed and accuracy throughout the network architecture.
Solution Approach 2:
The patent implements local quality by applying different quantization strategies to different layers of the neural network. Important layers that directly impact accuracy are quantized with higher precision, while non-critical layers use lower precision, thereby maintaining overall accuracy while improving operation speed and reducing memory usage.
2Manufacturing precision
If quantization aware training is performed to maintain accuracy, then learning data is needed, but privacy issues make data collection difficult
Solution Approach 1:
The patent uses copying by creating synthetic training data that mimics the characteristics of real user data without requiring actual user data collection. The system generates artificial datasets that preserve the statistical properties and patterns needed for quantization-aware training, thereby maintaining accuracy while avoiding privacy issues associated with collecting real user data.
3Manufacturing precision
If full precision floating point is used, then accuracy is maintained, but operation speed and memory usage are reduced
Solution Approach 1:
The patent applies parameter changes by transitioning from full precision floating-point representation to variable-precision integer representation. The system dynamically selects the appropriate bit width for each layer based on its importance, thereby significantly reducing memory usage while maintaining accuracy through selective preservation of precision in critical layers.
Solution Approach 2:
The patent extracts only the essential precision requirements from full floating-point representation. By identifying and retaining only the critical precision needs in important layers while using lower precision in non-critical layers, the system reduces overall memory usage while maintaining the accuracy necessary for proper neural network function.
Data Source
AI summary
An electronic apparatus includes: a communication interface; a memory storing at least one instruction; and a processor configured to execute the at least one instruction to: quantize a first neural network model for identifying an object, to acquire a second neural network model, control the communication interface to transmit information on the second neural network model to an external apparatus, receive, from the external apparatus, feature information for an image stored in the external apparatus acquired based on the second neural network model, and identification information corresponding to the feature information, and train the second neural network model based on the received feature information and the received identification information.