Systolic Array Neural Processor With On-Chip Buffering

The patent replaces software-based neural network computations with a dedicated hardware circuit that performs matrix multiplications and activation functions. This hardware implementation uses specialized computational units (matrix computation unit and vector computation unit) to execute neural network operations in parallel, achieving significant speed improvements while maintaining the ability to process different neural network architectures through reconfigurable data paths and control logic.

The patent implements a hierarchical memory architecture where activation inputs are stored in a first buffer and weight inputs are stored in a second buffer, both located on-chip. The matrix computation unit and vector computation unit access these buffers through integrated data paths, eliminating the need for repeated off-chip memory accesses during neural network computations. This nested buffer structure keeps frequently accessed data close to the computational units, minimizing communication latency.

The patent divides the neural network computation into two distinct functional segments: a matrix computation unit that handles weight-activation multiplications and accumulations, and a vector computation unit that applies activation functions to the accumulated values. This segmentation allows each unit to be optimized for its specific function, improving computational efficiency and reducing the energy required per operation while maintaining the ability to process large input sizes with high accuracy.