Neural Network Output Layer Reconfiguration for Compressed Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning-based neural networks face performance degradation due to compression, particularly affecting the accuracy of regression models like object detection, where output resolution decreases.
Innovation Solution
Transforming a regression problem into a classification problem by adjusting the network configuration, specifically increasing the number of nodes in the output layer and changing the distribution of nodes, while maintaining computational resource constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If scalar quantization is performed to compress the model, then the amount of computation is reduced, but the output resolution and accuracy of the model are degraded
Solution Approach 1:
The patent changes the fundamental parameter of the output representation from continuous regression values to discrete classification categories. By transforming the output layer to produce class probabilities instead of continuous values, the model achieves better precision under compression since classification problems are inherently more robust to quantization effects than regression problems.
Solution Approach 2:
Instead of trying to improve regression accuracy under compression, the patent inverts the approach by transforming the regression problem into a classification problem. This inversion leverages the natural robustness of classification to compression artifacts, thereby achieving better output resolution without requiring additional computational resources.
2Quantity of substance
If the model is compressed through quantization, then the model size is reduced, but the accuracy of outputs decreases
Solution Approach 1:
The patent changes the output parameter from continuous regression values to discrete classification labels. This parameter transformation makes the model more reliable under compression because classification outputs are less sensitive to the precision losses inherent in quantization, thereby maintaining accuracy while reducing model size.
Solution Approach 2:
The patent converts the harmful effect of compression (which degrades continuous value precision) into a benefit by transforming the problem type. Classification problems benefit from compression since discrete category assignments are more robust to the precision losses, thereby turning the compression artifact into an advantage for maintaining reliability.
3Productivity
If regression problem is solved with compressed model, then computational resources are saved, but output accuracy is degraded
Solution Approach 1:
The patent changes the nature of the output parameter from continuous regression values requiring high precision to discrete classification categories that are more tolerant of compression. This allows the model to maintain high output accuracy while operating efficiently with compressed weights, as classification tasks are inherently more robust to quantization errors.
Solution Approach 2:
Instead of attempting to solve the regression problem with higher precision under compression, the patent inverts the approach by converting the regression problem into a classification problem. This inversion achieves better accuracy-computation efficiency tradeoff because classification outputs are naturally more resilient to the precision limitations imposed by model compression.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing method including the following executed using a computer: obtaining a neural network model that solves a regression problem; obtaining input data and label data corresponding to the input data; compressing a network of the neural network model to obtain a compressed model; transforming the regression problem to be solved by the neural network model into a classification problem, based on information indicating performance of the compressed model, the information being calculated using the label data and output data which is obtained by inputting the input data to the compressed model; and changing a network configuration of the neural network model and transforming the label data, in accordance with the transformation from the regression problem to the classification problem.