Neural Network Projection Space for Low-Precision Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face limitations in processing large amounts of data quickly and accurately due to limited precision in connection weights, leading to reduced inference performance, especially when hardware or software constraints restrict network size and weight precision.
Innovation Solution
The proposed solution involves forming a projection space using output values from a neural network and reference vectors, allowing for linear processing and classification of new data without relying on gradient descent algorithms, thereby improving inference performance under limited conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the neural network processes large amounts of data with limited precision weights, then processing speed is improved, but inference accuracy deteriorates
Solution Approach 1:
The patent transforms the classification problem from the original high-dimensional feature space to a lower-dimensional projection space. By projecting neural network output values onto a projection space formed by reference vectors, the system reduces dimensionality while preserving essential classification information, thereby maintaining accuracy with reduced computational complexity
Solution Approach 2:
The patent introduces a projection space as an intermediary between the neural network output and the final classification decision. This projection space, formed by reference vectors learned through gradient descent, acts as a mediator that transforms high-precision neural network outputs into lower-precision representations that retain classification accuracy
2Device complexity
If the neural network uses quantized weights to reduce complexity, then device complexity is reduced, but inference performance deteriorates
Solution Approach 1:
The patent extracts the classification task from the full neural network processing pipeline and separates it into two stages: (1) neural network feature extraction with quantized weights, and (2) projection-based classification in a simplified projection space. This extraction allows the use of low-precision weights in the neural network while maintaining high classification performance through the projection mechanism
Solution Approach 2:
The patent changes the precision parameters of the neural network weights from high precision to low precision (quantized form), while compensating for the resulting performance loss by introducing a projection space with reference vectors that are learned to maintain classification accuracy despite the reduced weight precision
3Productivity
If the neural network is trained quickly with limited data, then training speed is improved, but learning accuracy deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-training the neural network on a large dataset to learn robust feature representations, then uses these pre-learned features for rapid adaptation to new tasks with limited data. The projection space and reference vectors are also learned in advance, enabling fast inference without requiring extensive task-specific training
Solution Approach 2:
The patent creates a universal neural network model that can be applied to multiple classification tasks. By learning a projection space that works across different tasks and data distributions, the system achieves good performance on new tasks with limited training data, as the projection mechanism generalizes well beyond the training distribution
Data Source
AI summary
A data processing method by learning of a neural network may be provided. The data processing method by the learning of a neural network includes: obtaining a first set of output values by processing a first set of input values of a task by the neural network; forming a projection space on the basis of the first set of output values; obtaining a second set of output values by processing a second set of input values out of input values of the task by the neural network; projecting the second set of output values onto the projection space; and performing processing the second set of output values in the projection space.