Tight-Coupled CPU Accelerator With Instruction Queue for ML Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional CPU architectures lack sufficient computing capacity, power efficiency, and programmability for machine learning workloads, while GPU and NPU architectures suffer from complicated software stacks and heterogeneous programming models.
Innovation Solution
A CPU instruction-based neural network accelerator is integrated with a general-purpose CPU core, providing high computing capacity, programmability, and power efficiency, with a separate acceleration data path for handling acceleration workloads, and results are stored in embedded memory for efficient computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If traditional CPU architectures are used for machine learning workloads, then programmability is maintained, but computing capacity and power efficiency are insufficient
Solution Approach 1:
The patent merges the CPU core with a neural network accelerator on the same chip, creating a hybrid architecture where the accelerator handles compute-intensive ML workloads while the CPU handles control and data preparation. This combining approach provides high computing capacity for ML tasks while maintaining the programmability of CPU architectures, resolving the contradiction between computing capacity and power efficiency.
Solution Approach 2:
The accelerator is designed to be tightly coupled with the CPU core, allowing it to process both dedicated neural network workloads and potentially other processing tasks. The shared memory and instruction queue enable the accelerator to function as an extension of the CPU, providing multi-functionality that maintains efficiency across different workload types while specialized for ML acceleration.
2Power
If GPU or NPU architectures are used for machine learning, then computing capacity is improved, but software stack complexity increases
Solution Approach 1:
The patent segments the processing system into distinct functional units: a CPU core for general-purpose computing and control, and a dedicated accelerator for neural network workloads. The accelerator includes its own instruction queue and embedded memory, creating modular functional blocks that simplify the software interface. This segmentation reduces software stack complexity by providing a clean separation of concerns while maintaining high computing capacity for ML tasks.
Solution Approach 2:
The patent introduces an instruction queue as an intermediary between the CPU and the accelerator. This instruction queue receives decoded instructions from the CPU and manages their execution on the accelerator, serving as a buffer and mediator that simplifies the interaction interface. The intermediary structure allows the CPU to continue executing general-purpose code while the accelerator handles ML workloads, reducing the complexity of software integration.
3Power
If a separate accelerator is added to the CPU, then computing capacity for machine learning is improved, but device complexity increases
Solution Approach 1:
The patent merges the accelerator with the CPU by integrating them on the same chip with shared memory and tightly coupled instruction queues. This physical integration reduces the overall device complexity compared to having completely separate accelerator chips, while still providing the computing capacity enhancement needed for machine learning workloads.
Solution Approach 2:
The accelerator is nested within the CPU architecture, with the instruction queue embedded in the CPU and the accelerator containing its own embedded memory. This nested structure allows the accelerator to function as an integrated component rather than a separate add-on, reducing device complexity while maintaining enhanced computing capacity for neural network processing.
4Quantity of substance
If data is transferred between CPU memory and accelerator memory, then data availability is improved, but data hazards and processing delays occur
Solution Approach 1:
The patent implements preliminary action by pre-loading data into the accelerator's embedded memory before the actual computation begins. The instruction queue manages data transfer in advance, and the accelerator can maintain its own data buffers. This preliminary data preparation reduces the need for critical data transfers during computation, minimizing data hazards and processing delays while ensuring data availability.
Data Source
AI summary
An integrated circuit includes: a central processing unit (CPU) core; an accelerator; and an acceleration instruction queue connected to the CPU core and the accelerator. The CPU core is to: fetch and decode one or more instructions from among an instruction sequence in a programmed order; determine an instruction from among the one or more instructions containing an acceleration workload encoded therein; and queue the instruction containing the acceleration workload encoded therein in the acceleration instruction queue.