Deep Neural Network Compression via Mask-Based Pruning

The patent applies preliminary action by performing sensitivity analysis and identifying important connections before pruning occurs. The method pre-processes the network to determine which connections are critical, then uses this information to guide the pruning process, ensuring that important connections are preserved while removing only truly redundant ones. This preliminary identification step prevents accuracy loss that would occur with random or purely weight-threshold-based pruning.

The patent implements feedback mechanisms through iterative pruning and retraining cycles. After each pruning operation, the network is retrained and evaluated, and the results feed back into the next pruning iteration. This feedback loop allows the system to adaptively adjust the pruning threshold and preserve connections that are important for maintaining accuracy, while progressively removing redundant connections to improve computational efficiency.

The patent applies local quality by treating different connections in the neural network differently based on their individual importance. Rather than uniformly pruning all connections below a global threshold, the method performs local sensitivity analysis to identify which specific connections are critical for accuracy. This allows the network to have varying densities across different regions, with important connections preserved and redundant ones removed, achieving compression without uniform accuracy loss.

The patent utilizes parameter changes by dynamically adjusting the pruning threshold and other compression parameters based on the specific network architecture, task requirements, and available resources. The method changes parameters such as the weight threshold for pruning, the sensitivity analysis depth, and the retraining learning rate to optimize the balance between compression ratio and accuracy retention for mobile deployment scenarios.

The patent applies partial action by performing sensitivity analysis on only a subset of connections or using approximate sensitivity metrics rather than exhaustive analysis. The retraining process is also performed for limited iterations rather than full convergence each time. This partial approach reduces the time cost of iterative pruning while still achieving sufficient accuracy preservation, accepting that not every connection needs perfect optimization.

The patent implements periodic action by performing full sensitivity analysis and retraining only at specific intervals or milestones during the compression process, rather than after every single pruning operation. Between these periodic full cycles, lighter pruning operations are performed with fewer retraining iterations. This periodic approach reduces overall training time while maintaining accuracy through the structured sequence of analysis-pruning-retraining cycles.