Processor Core Inter-Functional Unit Data Bypass for ML Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face inefficiencies and latency issues due to the need for frequent access to shared register blocks for data transfer between functional units, leading to performance bottlenecks in matrix vector operations, especially in machine learning and deep computing applications.
Innovation Solution
The implementation of a computing system with private vector registers for each functional unit and an inter-functional unit data bypass (IFUDB) block that routes signals directly between functional units without using shared or private memory blocks, enabling parallel data processing and transfer of results before all elements are computed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred between functional units through shared register blocks, then data transfer is enabled, but latency and performance bottlenecks occur due to frequent access to shared memory
Solution Approach 1:
The patent divides the monolithic shared register block into multiple private vector register blocks, each dedicated to specific functional units. This segmentation eliminates the need for functional units to contend for access to a single shared resource, thereby reducing latency and improving data transfer speed between connected functional units.
Solution Approach 2:
The patent introduces an inter-functional unit data bypass (IFUDB) block as an intermediary that enables direct data routing between functional units without requiring access to shared or private memory blocks. This bypass mechanism acts as a dedicated communication channel that eliminates memory access latency for intermediate data transfers.
2Quantity of substance
If shared vector registers are used for data storage, then memory capacity is maximized, but access conflicts and latency increase for multiple functional units
Solution Approach 1:
The shared vector register block is segmented into multiple private vector register blocks, each with dedicated capacity for specific functional units. This allows each functional unit to have sufficient local storage capacity while avoiding access conflicts, thereby maintaining both memory capacity and computational efficiency.
Solution Approach 2:
Each private vector register block is optimized for the specific functional units it serves, providing locally optimized data storage with immediate access. This local quality ensures that each functional unit has dedicated capacity without competing for shared resources, improving overall computational efficiency.
3Reliability
If all elements of a vector register are processed before routing results, then data completeness is ensured, but processing speed and throughput are reduced
Solution Approach 1:
The IFUDB block enables preliminary routing of computed elements before the entire vector register processing is complete. As soon as individual elements are computed by a functional unit, they can be immediately routed to subsequent functional units through the bypass, allowing overlapping execution and improving processing speed while maintaining data completeness through proper synchronization.
Data Source
AI summary
A computing system includes a plurality of functional units, each functional unit having one or more inputs and an output. There is a shared memory block coupled to the inputs and outputs of the plurality of functional units. There is a private memory block assigned to each of the plurality of functional units. An inter functional unit data bypass (IFUDB) block is coupled to the plurality of functional units. The IFUDB is configured to route signals between the one or more functional units without use of the shared memory block.