Hardware-optimized recurrent neural network system

FlashRNN optimizes RNNs on GPUs with block-diagonal matrices and fused kernels, addressing inefficiencies in traditional RNNs, achieving 50x speedups and enabling larger hidden sizes, enhancing parallel processing efficiency.

WO2026074103A1 Publication Date: 2026-04-09NXAI GMBH
1 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2025/078392
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-10-02
Filing Date
2025-10-02
Publication Date
2026-04-09

AI Technical Summary

Technical Problem

Traditional RNN architectures face inefficiencies in parallel processing due to their sequential nature, limiting their performance on modern hardware like GPUs, and there is a need for improved implementations that leverage parallel processors effectively while maintaining state-tracking capabilities.

Method used

The implementation of FlashRNN, which optimizes RNNs on GPUs by using a parallelization variant with block-diagonal recurrent weight matrices and fused computational kernels, leveraging on-chip memory and reduced precision formats like bfloat16, and employing optimization modules for efficient resource utilization.

Benefits of technology

FlashRNN achieves significant speedups, allowing 50x faster processing compared to vanilla implementations and enabling 32x larger hidden sizes, optimizing memory and computational efficiency on diverse hardware.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A system includes a machine-learning model implemented on a data processing apparatus, which features a parallel processor with a memory hierarchy. The machine-learning model is a recurrent neural network (RNN) with a multi-head architecture, comprising multiple sub- vectors that process parallel data streams. The RNN's weight matrix is structured as a block-diagonal matrix, allowing for parallel processing of the sub-vectors. A fused computational kernel executes an entire time-series processing loop for the multi-head RNN, maintaining the weight matrix blocks in on-chip memory and performing matrix multiplications and element-wise operations for each sub-vector in a single kernel execution.
Need to check novelty before this filing date? Find Prior Art