Parallel Memory Partitioning for Data Transfer Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face challenges in optimizing data transfers between parallel processing processors and external memory, particularly in deep neural networks and embedded systems, where limited resources and energy constraints complicate efficient data movement.
Innovation Solution
The proposed computing system partitions the external memory into multiple partitions, mirroring the column arrangement of the parallel processing processor's memories. This allows for simultaneous data transfers across multiple columns via dedicated interface links, increasing transfer rate and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data transfers are performed sequentially through a single interconnection bus, then the system structure remains simple, but the data transfer rate is limited and becomes a performance bottleneck
Solution Approach 1:
The external memory is divided into multiple partitions (e.g., Memory A0, Memory A1, Memory A2, Memory A3) that can be accessed simultaneously by different computing units. This segmentation allows parallel data transfers through dedicated interface links, dramatically increasing the overall data transfer rate without creating a single-point bottleneck
Solution Approach 2:
The patent introduces a column-based organizational dimension where computing units are arranged in columns and each column has dedicated interface links to memory partitions. This dimensional organization enables simultaneous data transfers across multiple columns, transforming the single-bus limitation into a multi-path parallel architecture
2Productivity
If the host processor manages all data transfers, then the system control is centralized and simple, but the host processor becomes a bottleneck and cannot perform other critical tasks
Solution Approach 1:
Computing units are equipped with direct memory access capabilities through dedicated interface links, allowing them to autonomously transfer data between their local memory (Memory B) and external memory partitions without requiring host processor intervention. This self-service mechanism frees the host processor to focus on coordinate-level tasks while data movement occurs independently at the computing unit level
Solution Approach 2:
The patent introduces memory access control modules as intermediaries that manage data transfers between computing units and memory partitions. These control modules handle the complexity of coordinated access, arbitration, and synchronization, allowing the host processor to remain simple while enabling sophisticated parallel data transfer operations
3Productivity
If memory access is centralized through a single interface, then the memory control logic is simple, but the data transfer efficiency is limited by the single access path
Solution Approach 1:
The memory interface is segmented into multiple dedicated interface links, each connecting a column of computing units to specific memory partitions. This segmentation creates multiple independent data pathways that can operate simultaneously, multiplying the effective data transfer efficiency by the number of parallel links
Solution Approach 2:
Each computing unit's interface link serves multiple functions: it acts as a data transfer channel, an address routing path, and a control signal conduit. This multi-functionality allows the same physical interface infrastructure to support complex parallel operations without proportionally increasing hardware complexity
Data Source
AI summary
Computing system comprising a memory A, a memory access control module, and a parallel processing processor comprising a plurality of computing units each comprising a memory B shared by one or more elementary processors. The computing units are arranged in columns. A partition of the memory A is associated with each column. Connection modules are connected respectively to the partitions of the memory A and to the memories B of the computing units. Within a column, the neighboring connection modules are connected to one another by a dedicated interface link. The memory access control module is adapted to configure the connection modules to execute parallel data transfers in a plurality of columns, each transfer making it possible to exchange data within a column between a partition of the memory A and a memory B of at least one computing unit.