Mixed-Scale Instruction Dispatch for Heterogeneous SoC IP Cores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing hardware heterogeneity in system on chips (SoCs) due to domain-specific architectures leads to low programming productivity and hardware utilization, as existing SoCs manage heterogeneity through a host CPU, resulting in non-negligible overheads and inefficient software management.
Innovation Solution
A unified system on chip architecture (SaaP) that manages hardware heterogeneity through an IP-level pipeline, using mixed-scale instructions to hide hardware differences and enable efficient execution across various IP cores, with a system controller handling instruction fetching, decoding, and dispatching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a host CPU manages hardware heterogeneity through code running on the host CPU, then hardware interfaces can be exposed, but programming productivity decreases and hardware utilization becomes inefficient due to non-negligible overheads in control and data exchange
Solution Approach 1:
The patent introduces an instruction dispatcher as an intermediary component that sits between the instruction fetcher/decoder and the heterogeneous IP cores. This dispatcher translates unified instructions into IP-specific operations, eliminating the need for host CPU management code and programming frameworks. The intermediary handles the complexity of hardware heterogeneity internally, allowing application developers to write code once that runs efficiently across different IP types without manual hardware management.
Solution Approach 2:
The patent creates a universal instruction set that can be executed by multiple types of IP cores (GPU, NPU, DPU, etc.). Instead of requiring separate management code for each hardware type, the unified instruction architecture allows a single instruction format to be dispatched to different IP cores based on the operation type, eliminating the need for programming frameworks like PyTorch or TensorFlow to implement separate high-level interfaces for each IP.
2Adaptability or versatility
If a host CPU manages hardware heterogeneity through code running on the host CPU, then hardware interfaces can be exposed, but hardware utilization becomes inefficient due to non-negligible overheads in control and data exchange
Solution Approach 1:
The instruction dispatcher serves as an on-chip intermediary that eliminates the need for data to be transferred to and from the host CPU for management operations. All instruction translation and IP coordination happens locally on the chip, reducing control and data exchange overheads between the host CPU and heterogeneous IPs, thereby improving hardware utilization and reducing energy loss.
3Ease of operation
If programming frameworks implement high-level abstract interfaces using different IPs, then application developers can use Python interfaces, but development work increases significantly for framework developers
Solution Approach 1:
The patent implements a universal instruction architecture where a single instruction set can be executed by multiple types of IP cores (GPU, NPU, DPU, etc.). This eliminates the need for programming frameworks to implement separate high-level abstract interfaces for each IP type. The unified instruction format handles different IP operations natively, reducing framework development work while maintaining ease of use for application developers.
4Reliability
If the same application requires different implementations for different IPs, then hardware-specific optimization is possible, but programming burden increases heavily for application developers
Solution Approach 1:
The patent creates a universal instruction set that can be executed by different IP cores while maintaining hardware-specific optimization capabilities. The instruction dispatcher automatically routes instructions to the appropriate IP type based on the operation, allowing application developers to write a single implementation that is optimized for each hardware type without manually creating different versions for GPU, NPU, DPU, etc.
Data Source
AI summary
The present application provides an instruction processing apparatus, an instruction execution method, a system on chip and a board card. The solution described in the present application may hide the heterogeneity of execution units by proposing a unified mixed-scale instruction, thereby improving programming efficiency and hardware utilization.