On-Chip Heterogeneous AI Processor with Distributed Task Queues

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current AI processor architectures, such as GPU, FPGA, and ASIC, face inefficiencies due to high power consumption, limited scalability, and poor adaptability, leading to bottlenecks in AI computation tasks and increased costs, especially when handling diverse AI applications.

Innovation Solution

An on-chip heterogeneous AI processor is developed, incorporating multiple architectural types of computation units like ASIC, GPGPU, and FPGA, with a controller to manage and schedule tasks efficiently, supporting independent, cooperative, and interactive operation modes, and an access control module for data management, to optimize AI computation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a multi-core CPU is used to compensate for system performance, then computation performance is improved, but power consumption and cost increase

Engineering Contradiction:

Improvecomputation performanceVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system segments computation tasks into different types (AI computation vs. other computations) and assigns them to specialized computation units. The AI computation unit handles AI-specific tasks while the CPU handles other tasks, avoiding the need for all CPU cores to run at high performance for all workloads.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a dedicated AI computation unit with specialized architecture optimized for AI workloads, rather than relying on general-purpose CPU cores. This local specialization allows efficient AI computation without requiring the entire CPU system to be over-provisioned.

Inventive Principle:

Principle #3Local quality

2Productivity

If a heterogeneous architecture with CPU and AI chip is used, then AI computation capability is improved, but coordination efficiency deteriorates due to CPU bottleneck

Engineering Contradiction:

ImproveAI computation capabilityVSAvoidcoordination efficiency

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent merges the AI computation unit and CPU into a single integrated system with unified memory access and coordinated task scheduling. This integration eliminates the communication overhead and coordination inefficiencies associated with separate AI chip and CPU architectures.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system introduces a task management mechanism that acts as an intermediary between the CPU and AI computation unit, efficiently scheduling and coordinating tasks between the two computation types without requiring complex inter-processor communication protocols.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If GPU is used for AI computation, then parallel computation capability is improved, but chip size and power consumption increase

Engineering Contradiction:

Improveparallel computation capabilityVSAvoidchip size

Core Design Contradiction:

ProductivityVSArea of stationary object

Solution Approach 1:

The patent designs a dedicated AI computation unit with architecture specifically optimized for AI workloads, providing GPU-like parallel computation capabilities but with reduced chip area by eliminating unnecessary general-purpose computing components and using more efficient memory structures.

Inventive Principle:

Principle #3Local quality

4Use of energy by moving object

If ASIC is used for AI processing, then power consumption and chip area are reduced, but adaptability and scalability deteriorate

Engineering Contradiction:

Improvepower consumptionVSAvoidadaptability

Core Design Contradiction:

Use of energy by moving objectVSAdaptability or versatility

Solution Approach 1:

The system implements a dynamic task scheduling mechanism that can allocate computation tasks to different computation units based on workload characteristics. The AI computation unit can be dynamically activated or deactivated, and task priorities can be adjusted, providing adaptability while maintaining the energy efficiency of specialized hardware.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11789895B2On-chip heterogeneous AI processor with distributed tasks queues allowing for parallel task execution

Publication Date: 2023.10.17 SUZHOU DENGLIN TECHNOLOGIES CO LTD

AI summary

Embodiments described herein provide an on-chip heterogeneous Artificial Intelligence (AI) processor comprising at least two different architectural types of computation units, wherein each of the computation units is associated with a respective task queue configured to store computation subtasks to be executed by the computation unit. The AI processor also comprises a controller configured to partition a received computation graph associated with a neural network into a plurality of computation subtasks according to a preset scheduling strategy and distribute the computation subtasks to the task queues of the computation units. The AI processor further comprises a storage unit configured to store data required by the computation units to execute their respective computation subtasks and an access interface configured to access an off-chip memory. Different application tasks are processed by managing and scheduling the different architectural types of computation units in an on-chip heterogeneous manner.