Cross-Core Copy Instructions for Operand Transfer Between On-Chip Memories
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional acceleration units face inefficiencies in operand transfer due to limited on-chip memory, leading to low performance as operands often need to be moved from on-chip memory to shared memory outside the core, resulting in degraded performance.
Innovation Solution
The acceleration unit incorporates a primary core and secondary core with on-chip memories, utilizing cross-core copy instructions and synchronization primitives to efficiently transfer operands between cores, optimizing data movement within on-chip memories rather than to external shared memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If operands are stored in on-chip memory of a single core, then access speed is improved, but memory capacity is limited and cannot accommodate all operands
Solution Approach 1:
The system divides the memory storage function across multiple cores. Each core has its own on-chip memory (segmented memory spaces), allowing the acceleration unit to collectively accommodate more operands than a single core could hold, while each core maintains fast access to its local memory.
Solution Approach 2:
The patent transitions from a single-core memory model to a multi-core memory model, adding the dimension of core distribution. Operands are distributed across multiple on-chip memories of different cores, effectively expanding the total accessible memory capacity while maintaining fast on-chip access speeds.
2Quantity of substance
If operands are transferred to shared memory outside the core, then memory capacity is increased, but transfer efficiency deteriorates
Solution Approach 1:
The patent merges the on-chip memories of multiple cores into a unified memory system accessible through cross-core copy operations. This allows operands to be shared between cores via fast on-chip interconnects rather than slow external memory access, combining the capacity benefits of multiple cores with the speed benefits of on-chip access.
Solution Approach 2:
The patent introduces cross-core copy instructions as an intermediary mechanism that enables efficient operand transfer between cores' on-chip memories. This intermediary mechanism avoids the need for slow external shared memory access while still allowing multiple cores to access the same operands.
3Productivity
If multiple cores perform the same model task in parallel, then processing capacity is improved, but memory resources are not utilized efficiently for different tasks
Solution Approach 1:
The patent makes the on-chip memory system universal by enabling it to serve multiple purposes: each core can execute different model tasks with different operands simultaneously. The cross-core copy mechanism allows any core to access operands from any other core's memory, creating a versatile system that can handle diverse workloads while maintaining parallel processing capability.
Data Source
AI summary
An acceleration unit including a primary core and a secondary core is provided. The primary core includes a first on-chip memory, a primary core sequencer adapted to decode a received first cross-core copy instruction, and a primary core memory copy engine adapted to acquire a first operand from a first address in the first on-chip memory and copy the acquired first operand to a second address in a second on-chip memory of the secondary core. Further, the secondary core includes a second on-chip memory, a secondary core sequencer adapted to decode a received second cross-core copy instruction, and a secondary core memory copy engine adapted to acquire the first operand from the second address in the second on-chip memory and copy the acquired first operand back to the first address in the first on-chip memory.