Multi-port Vector Register File for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer architectures struggle to efficiently compute both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) simultaneously, often specializing in one type and compromising on performance for the other, which is exacerbated by real-time latency requirements.
Innovation Solution
A processor architecture featuring a multi-port vector register file and a CNN-aware instruction set architecture that enables efficient matrix-vector multiplication, allowing for high throughput and low latency by leveraging parallelism in CNNs and supporting RNNs through a system of quasi-independent matrix-vector multiplication tile engines and a dual-port memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a processor architecture specializes in either CNNs or RNNs, then performance for that specific network type is improved, but performance for the other network type deteriorates
Solution Approach 1:
The patent implements a universal processor architecture that can efficiently execute both CNN and RNN operations through a single instruction set architecture. The vector register file with multi-port memory and tile engine design enables the same hardware to perform convolution operations for CNNs and recurrence operations for RNNs, eliminating the need for separate specialized processors and achieving high performance for both network types simultaneously
2Loss of time
If real-time latency requirements are imposed, then response time is improved, but computational complexity increases
Solution Approach 1:
The patent divides the processing architecture into multiple independent tile engines that can operate in parallel. Each tile engine is a simplified processing unit that handles specific portions of the computation. This segmentation allows the system to achieve low latency through parallel execution while keeping each individual tile engine relatively simple in structure, avoiding the need for overly complex sequential processing
3Productivity
If parallelism is leveraged to improve throughput, then processing speed is improved, but memory access complexity increases
Solution Approach 1:
The patent combines multiple memory ports into a unified multi-port memory structure within the vector register file. Instead of having separate memory systems for different data types or access patterns, the design merges read and write ports into a single coordinated memory subsystem. This consolidation enables parallel memory access for throughput improvement while using unified control logic to manage access complexity, rather than requiring multiple independent memory controllers
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Neural network processors including a vector register file (VRF) having a multi-port memory and related methods are provided. The processor may include tiles to process an N by N matrix of data elements and an N by 1 vector of data elements. The VRF may, in response to a write instruction, store N data elements in a multi-port memory and during each one of out of P clock cycles provide N data elements to each one of P input interface circuits of the multi-port memory comprising an input lane configured to carry L data elements in parallel. During the each one of the P clock cycles the multi-port memory may be configured to receive N data elements via a selected at least one of the P input interface circuits. The VRF may include output interface circuits for providing N data elements in response to a read instruction.