Hierarchical CPU Tiles for Parallel ML Workload Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data processing systems face inefficiencies in handling computationally-intensive tasks, particularly machine learning workloads, due to the limitations of general-purpose CPUs and the need for centralized control of hardware accelerators, which leads to performance overheads and resource contention.

Innovation Solution

A compute system with a CPU hierarchy and a cluster of compute tiles, each comprising a tile CPU and a hardware accelerator, allows for parallel processing and decentralized control of accelerators, reducing overheads and improving performance through dedicated interfaces and shared memory management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a general-purpose CPU is used to handle computationally-intensive tasks, then versatility is maintained, but processing speed and efficiency deteriorate

Engineering Contradiction:

ImproveCPU versatilityVSAvoidprocessing speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system segments processing tasks by introducing multiple CPU levels (first-level, second-level, third-level CPUs) with specialized functions. The first-level CPU handles general-purpose tasks while second and third-level CPUs handle computationally-intensive tasks, allowing the system to maintain versatility through the first-level CPU while improving processing speed through specialized lower-level CPUs.

Inventive Principle:

Principle #1Segmentation

2Device complexity

If centralized control of hardware accelerators is implemented, then resource management is simplified, but performance overhead and resource contention increase

Engineering Contradiction:

Improveresource management complexityVSAvoidperformance overhead

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system applies local quality by granting specific resource management authorities to different CPU levels. The first-level CPU manages general resources, while second and third-level CPUs have localized control over hardware accelerators and computationally-intensive tasks respectively. This distributed authority reduces performance overhead by allowing local decision-making without requiring centralized coordination for every resource allocation.

Inventive Principle:

Principle #3Local quality

3Productivity

If a multi-level CPU hierarchy is introduced to improve processing performance, then productivity increases, but device complexity increases

Engineering Contradiction:

Improveprocessing performanceVSAvoidCPU hierarchy complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system resolves complexity by organizing CPUs in a hierarchical dimension rather than a flat structure. The first-level, second-level, and third-level CPUs are arranged in vertical layers with clear communication protocols between levels. This dimensional organization allows the system to achieve high processing performance through specialized processing paths while managing complexity through structured inter-level communication rather than complex peer-to-peer interactions.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS20260037301A1Compute system

Publication Date: 2026.02.05 ARM LTD

AI summary

A compute system comprises a CPU (central processing unit) hierarchy comprising: a first-level CPU; a second-level CPU; and a plurality of third-level CPUs.