Layered Memory Architecture for Neural Network Data Exchange

The patent divides the centralized shared memory into multiple memory banks (first memory bank, second memory bank, etc.) that can be accessed independently by different SIMD lanes. This segmentation allows parallel memory access operations, reducing bank conflicts and lowering access latency while maintaining data exchange capability between lanes.

The patent introduces a memory controller as an intermediary between SIMD lanes and memory banks. The memory controller manages memory access requests, allocates banks to different lanes, and coordinates read/write operations. This intermediary enables efficient memory access with reduced latency while avoiding the need for expensive exponential cross-bar connections, as the controller serializes and manages access to shared memory resources.

The patent designs memory banks with multi-functional capabilities where each bank can serve multiple SIMD lanes dynamically. The memory controller implements bank allocation strategies that assign different banks to different lanes based on access patterns, allowing fewer physical memory banks to serve the function of handling conflicts that would otherwise require many more dedicated banks. This universal approach improves data exchange efficiency while controlling hardware resource consumption.