Semantic Masking for Transformer Model Compression

The patent extracts and retains only the most important hidden layers from the pre-trained transformer model based on semantic probing performance scores. By selecting only the top-performing layers (e.g., layers with highest semantic probing scores) and discarding less important layers, the model size is significantly reduced while maintaining core language understanding capabilities. This extraction approach directly addresses the contradiction by keeping essential functionality while removing redundant components.

The patent segments the transformer model into individual hidden layers that can be independently evaluated and selected. Instead of treating the model as an indivisible unit, it divides the model into discrete layers, probes each layer's semantic performance, and then reconstructs the model using only the selected layers. This segmentation enables fine-grained optimization of model size versus performance trade-off.

The patent performs preliminary semantic probing on a pre-trained model to identify and select the most important hidden layers before actual on-device training. By pre-determining which layers are most useful for language understanding through probing, the subsequent on-device training only needs to train the selected layers rather than the entire model, significantly reducing training time and computational resources required on constrained devices.

The patent extracts only the essential hidden layers that contribute most to language understanding performance and excludes less important layers from on-device training. This extraction reduces the number of parameters that need to be trained on-device, directly reducing training time and energy consumption while maintaining language processing capability.

The patent extracts and retains only the most semantically important hidden layers for on-device inferencing based on probing results. By removing less important layers, the model size for deployment is significantly reduced, which decreases memory usage, reduces processing complexity, and improves energy efficiency while maintaining core language understanding accuracy through retention of critical layers.

The patent applies local quality by differentiating the importance of different model components. Instead of treating all layers uniformly, it identifies which specific hidden layers have high semantic probing scores and retains only those for deployment. This creates a non-uniform model structure where critical components are preserved and less critical components are removed, optimizing the balance between accuracy and processing efficiency.