Block-Pruned RNN Inference for Real-Time Mobile Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for accelerating recurrent neural networks (RNNs) on mobile devices face challenges in achieving real-time inference due to high computation complexity and memory consumption, with non-structured pruning causing inefficiencies in hardware implementations and structured pruning leading to accuracy degradation.

Innovation Solution

A block-based structured pruning approach combined with compiler-assisted optimizations, including matrix reorder, load redundancy elimination, and a compact data format, to maintain high inference accuracy and reduce model size.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Volume of stationary object

If block-based structured pruning is applied to compress RNN models, then model size is reduced, but inference accuracy may deteriorate

Engineering Contradiction:

Improvemodel sizeVSAvoidinference accuracy

Core Design Contradiction:

Volume of stationary objectVSMeasurement precision

Solution Approach 1:

The weight matrix is divided into multiple blocks, and pruning is applied at the block level rather than individual weight level. This segmentation allows the model to maintain important weight patterns while removing redundant blocks, achieving compression without significant accuracy loss

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different pruning ratios are applied to different blocks based on their importance. Critical blocks retain more weights while less important blocks are compressed more aggressively, preserving local quality where needed while achieving global compression

Inventive Principle:

Principle #3Local quality

2Device complexity

If existing model compression techniques are applied to RNNs, then computation complexity is reduced, but hardware acceleration compatibility is lost

Engineering Contradiction:

Improvecomputation complexityVSAvoidhardware acceleration compatibility

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The pruning is applied to transform the RNN weight matrices into a structured sparse format with specific parameter patterns that are compatible with hardware acceleration. By changing the parameter distribution and structure rather than just reducing quantity, the model achieves both compression and hardware friendliness

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The compressed RNN model combines structured sparsity with compiler optimizations to create a composite representation that leverages both algorithmic compression and hardware capabilities, achieving compatibility with accelerated processors

Inventive Principle:

Principle #40Composite materials

3Ease of operation

If RNN inference is executed on mobile devices, then portability is improved, but energy consumption increases

Engineering Contradiction:

ImproveportabilityVSAvoidenergy consumption

Core Design Contradiction:

Ease of operationVSUse of energy by moving object

Solution Approach 1:

Redundant computations and memory access operations are extracted and eliminated from the RNN inference process. By removing unnecessary computational steps, the energy consumption per inference is reduced while maintaining the portability benefit of running on mobile devices

Inventive Principle:

Principle #2Taking out (Extraction)

4Productivity

If RNN inference is executed on mobile devices, then real-time processing capability is improved, but memory consumption increases

Engineering Contradiction:

Improvereal-time processing capabilityVSAvoidmemory consumption

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The compressed model structure nests sparse weight representations within optimized computation patterns, allowing the system to maintain real-time processing capability while reducing the actual memory footprint through efficient storage of only essential parameters

Inventive Principle:

Principle #7Nested doll (Nesting)

Data Source

PatentUS12626131B2Computer-implemented methods and systems for compressing recurrent neural network (RNN) models and accelerating RNN execution in mobile devices to achieve real-time inference

Publication Date: 2026.05.12 NORTHEASTERN UNIV (US)

AI summary

A recurrent neural network (RNN) acceleration framework leverages both a block-based pruning approach and compiler optimizations to accelerate RNN inference on mobile devices.