Exponential Model Embedding Mapping for Neural Network Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine-learned models, such as neural networks, face challenges in achieving compression gains, especially in the output layer when dealing with a large number of classes, leading to high compute and memory overhead, which limits their applicability on resource-constrained devices.
Innovation Solution
A machine-learned classification model combining an embedding model and an exponential model, where the exponential model applies a mapping between the embedding parameters and output classes, reducing the number of parameters needed for classification, allowing for significant compression gains by leveraging human knowledge and automatic feature learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network uses a large number of parameters to achieve high accuracy in classification tasks with many discrete classes, then classification accuracy is improved, but memory overhead and compute requirements increase significantly
Solution Approach 1:
The patent segments the classification task into two distinct stages: (1) an embedding model that maps input data to a compact embedding space with far fewer dimensions than the number of output classes, and (2) an exponential model that operates on this compressed embedding to produce final class probabilities. This segmentation allows the system to maintain high classification accuracy while dramatically reducing the number of parameters needed, as the embedding dimensionality is much smaller than the vocabulary size in tasks like next-word prediction
Solution Approach 2:
The patent introduces an embedding space as an intermediary representation between the input data and the final classification output. Instead of directly mapping input features to all possible output classes, the embedding serves as a compressed intermediate representation that captures essential information in fewer dimensions. This intermediary structure enables the system to achieve accurate classification with significantly fewer parameters than a direct mapping would require
2Speed
If a machine-learned model is deployed on resource-constrained devices like smartphones, then response time and security are improved, but the model size and compute requirements become prohibitive
Solution Approach 1:
The patent segments the model architecture to separate the heavy feature extraction operations (performed during training to create the embedding) from the lighter classification operations (performed during inference on resource-constrained devices). The embedding model produces a compact representation that requires minimal parameters to store and process, enabling deployment on mobile devices while maintaining accuracy
Solution Approach 2:
The patent fundamentally changes the parameter scale by transitioning from a direct classification approach requiring O(V) parameters (where V is vocabulary size) to an embedding-based approach requiring O(d) parameters where d << V. This parameter reduction through dimensional transformation makes it feasible to deploy high-accuracy models on resource-constrained devices with limited memory and compute capabilities
Data Source
AI summary
Aspects of the present disclosure enable humanly-specified relationships to contribute to a mapping that enables compression of the output structure of a machine-learned model. An exponential model such as a maximum entropy model can leverage a machine-learned embedding and the mapping to produce a classification output. In such fashion, the feature discovery capabilities of machine-learned models (e.g., deep networks) can be synergistically combined with relationships developed based on human understanding of the structural nature of the problem to be solved, thereby enabling compression of model output structures without significant loss of accuracy. These compressed models provide improved applicability to “on device” or other resource-constrained scenarios.