Non-uniform Compute Device Reordering for Data Locality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data processing systems with uniform compute devices face inefficiencies due to delayed processing caused by data retrieval from shared resources and poor utilization of host processor resources, especially when data locality is poor, and lack speculative execution capabilities.

Innovation Solution

A Non-uniform Compute Device (NUCD) architecture that integrates near-memory processing units as remote functional units of host processors, allowing selective routing of instructions to either host or near-memory processing units based on availability and data location, with mechanisms for speculative execution and data coherence maintenance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data is retrieved from shared data resources or lower level cache, then data availability is ensured, but processing is delayed

Engineering Contradiction:

Improvedata availabilityVSAvoidprocessing delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by checking data availability in higher level caches before initiating processing. The scheduler proactively determines data location and prepares data transfer paths in advance, preventing processing delays caused by data unavailability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a new dimension to data access by implementing near-memory processing units that can access data from multiple cache levels simultaneously. This multi-level cache architecture adds spatial and temporal dimensions to data retrieval, allowing parallel access paths that reduce latency while ensuring data availability.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If PIM instructions are executed in memory, then processing can proceed without waiting for data movement, but host processor resources are not fully utilized

Engineering Contradiction:

Improveprocessing throughputVSAvoidhost processor utilization

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The scheduler dynamically adapts instruction execution location based on real-time system state. When host processor resources are available, instructions are routed to the host for full resource utilization. When host resources are busy or data locality is poor, instructions are dynamically redirected to near-memory processing units, ensuring both productivity and adaptability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system implements multi-functionality by allowing the same instruction stream to be executed by different processing units (host processor or near-memory processing units) depending on conditions. This universal execution capability ensures that host processor resources are fully utilized when available, while maintaining processing throughput through alternative execution paths.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Loss of energy

If near-memory processing units are used, then energy and latency for data movement are reduced, but system complexity increases

Engineering Contradiction:

Improvedata movement energyVSAvoidsystem architecture complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The patent merges near-memory processing units with the existing memory hierarchy, integrating them as extension of the cache system rather than separate components. This merging approach reduces system complexity by reusing existing memory controllers, interconnects, and management logic, while still achieving energy and latency benefits of near-memory processing.

Inventive Principle:

Principle #5Merging (Combining)

4Productivity

If instructions are routed selectively to host or near-memory processing units, then processing efficiency is improved, but scheduling complexity increases

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidscheduling complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The scheduler implements self-service mechanisms by maintaining local knowledge of data location in the cache hierarchy and processing unit availability. Rather than requiring complex centralized arbitration, each scheduling decision is made autonomously based on pre-fetched data location information and simple availability checks, reducing scheduling complexity while maintaining high processing efficiency.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS10445094B2Method and apparatus for reordering in a non-uniform compute device

Publication Date: 2019.10.15 ARM LTD

AI summary

A data processing apparatus includes a multi-level memory system, one or more first processing unit coupled to the memory system at a first level and one or more second processing units each coupled to the memory system at a second level. A first reorder buffer maintains data order during execution of instructions by the first and second processing units and a second reorder buffer maintains data order during execution of the instructions by an associated second processing unit. An entry in the first reorder buffer is configured, dependent upon an indicator bit, as an entry for a single instruction or a pointer to an entry in the second reorder buffer. An entry in the second reorder buffer includes instruction block start and end addresses and indicators of input and output register. Instructions are released to a processing unit when all inputs, as indicated by the reorder buffers, are available.