Neural Network Embedding Weight Measurement via Masking

The patent applies local quality by assigning different importance weights to different entities within the embedding matrix. Instead of uniform quantization or approximation, the method identifies specific entities that are more important for the task and preserves their precision while allowing less important entities to be compressed. This is achieved through gradient-based optimization that learns entity-specific importance scores, enabling differential treatment of embedding elements based on their local significance to model performance.

The patent applies preliminary action by pre-computing and storing importance weights for each entity in the embedding matrix before the actual inference process. These importance weights are learned during a training phase using gradient descent, and once computed, they remain fixed or are updated sparingly. This preliminary computation of importance scores allows the model to quickly apply learned importance-based compression during inference without complex real-time calculations, achieving both efficiency and task-specific performance.

The patent applies dynamics by implementing a two-stage approach where the mask vector evolves from continuous values during training to binary values during inference. During the training phase, the mask vector contains continuous values that are optimized via gradient descent to accurately reflect entity importance. At inference time, a thresholding operation converts these continuous values to binary decisions (keep or prune). This dynamic transformation allows the system to enjoy the benefits of continuous optimization during learning while producing discrete, actionable results during deployment.