On-Chip Memory System for Reconfigurable Parallel Processor

Solution Overview

Problem

Existing on-chip memory systems for SIMT processors face challenges in optimizing area, delay, and power consumption due to extensive routing requirements for memory access, which complicates timing closure and dynamic switching power consumption.

Innovation Solution

A centralized-column approach is implemented, where columns of vector processing units are arranged in an array with memory units divided on either side, and a temporary storage buffer is used to create a circular data flow with output FIFOs driving succeeding columns, optimizing column-to-column interfaces and reducing power consumption by aligning buses bit-by-bit and interspersing them along column edges.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If extensive routing is provided to enable memory access in a processor with many processing elements, then memory access capability is improved, but timing closure and dynamic switching power consumption are worsened

Engineering Contradiction:

Improvememory access capabilityVSAvoiddynamic switching power consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by stationary object

Solution Approach 1:

The processor is divided into multiple columns, each with its own dedicated memory interface and cache blocks. This segmentation allows each column to access memory independently through localized interfaces, eliminating the need for extensive cross-chip routing while maintaining full memory access capability for all processing elements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from a two-dimensional array of processing elements to a three-dimensional architecture by stacking columns vertically and providing memory access from multiple sides. This dimensional change enables short routing distances within each column while maintaining comprehensive memory access capability across all processing elements.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If extensive routing is provided to enable memory access in a processor with many processing elements, then memory access capability is improved, but timing closure is worsened

Engineering Contradiction:

Improvememory access capabilityVSAvoidtiming closure

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The processor is divided into multiple columns, each with its own dedicated memory interface and cache blocks. This segmentation allows each column to access memory independently through localized interfaces, eliminating the need for extensive cross-chip routing while maintaining full memory access capability for all processing elements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Each column is equipped with localized memory interfaces and cache blocks positioned adjacent to the processing elements in that column. This local quality principle ensures that memory access routing is minimized within each column, improving timing closure while maintaining comprehensive memory access capability.

Inventive Principle:

Principle #3Local quality

3Area of stationary object

If columns are arranged to optimize area efficiency with short routing distances, then area and power consumption are improved, but routing complexity increases

Engineering Contradiction:

Improvechip area efficiencyVSAvoidrouting complexity

Core Design Contradiction:

Area of stationary objectVSDevice complexity

Solution Approach 1:

The processor is divided into multiple columns, each with its own dedicated memory interface and cache blocks. This segmentation allows each column to access memory independently through localized interfaces, eliminating the need for extensive cross-chip routing while maintaining full memory access capability for all processing elements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from a two-dimensional array of processing elements to a three-dimensional architecture by stacking columns vertically and providing memory access from multiple sides. This dimensional change enables short routing distances within each column while maintaining comprehensive memory access capability across all processing elements.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12093214B2On-chip memory system for a reconfigurable parallel processor

Publication Date: 2024.09.17 AZURENGINE TECH INC

AI summary

A processor may include a plurality of columns of vector processing units arranged in a two-dimensional column array with a plurality of column stacks placed side-by-side in a first direction and each column stack having two columns stacked in a second direction. The processor may further include a memory unit divided into two portions placed on two opposite sides of the column array in the second direction. Each portion may contain two memory blocks placed side-by-side in the first direction. Each memory block may contain two cache blocks placed along a first edge abutting an adjacent memory block and a plurality banks of memory cells placed to space from the first edge in the first direction by the two cache blocks and from a second edge abutting the column array in the second direction by routing channels.