Memory Page Access Instrumentation for Energy Reduction

Solution Overview

Problem

The increasing disparity between compute throughput and memory bandwidth limits the performance of memory access-intensive applications, as conventional solutions rely on explicit data placement and program execution management, which consume energy and memory bandwidth.

Innovation Solution

The generation of memory access profiles to co-locate data near processing units, reducing memory access energy by minimizing distances to access remote data, achieved through partitioning and mapping execution thread arrays and memory pages across processing units, using bipartite undirected graphs to minimize inter-partition traffic.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If data is explicitly migrated between processing units or local memories to co-locate data, then memory access performance is improved, but memory bandwidth and energy are consumed

Engineering Contradiction:

Improvememory access performanceVSAvoidenergy consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary profiling of memory access patterns during program execution to identify which data pages are frequently accessed by which processing units. This advance information is then used to guide data placement decisions, allowing the system to proactively co-locate data before access occurs rather than migrating data reactively, thereby reducing unnecessary data movement and energy consumption.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system enables processing units to self-optimize their data access by providing them with profiles of their own memory access patterns. Each processing unit can independently determine which data pages it frequently accesses and request their placement in its local memory, eliminating the need for centralized data migration management and reducing the energy overhead of explicit data movement operations.

Inventive Principle:

Principle #25Self-service

2Speed

If data is explicitly migrated between processing units or local memories to co-locate data, then memory access performance is improved, but memory bandwidth is consumed

Engineering Contradiction:

Improvememory access performanceVSAvoidmemory bandwidth

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The system performs preliminary profiling of memory access patterns during program execution to identify which data pages are frequently accessed by which processing units. This advance information is then used to guide data placement decisions, allowing the system to proactively co-locate data before access occurs rather than migrating data reactively, thereby reducing unnecessary data movement and energy consumption.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system enables processing units to self-optimize their data access by providing them with profiles of their own memory access patterns. Each processing unit can independently determine which data pages it frequently accesses and request their placement in its local memory, eliminating the need for centralized data migration management and reducing the energy overhead of explicit data movement operations.

Inventive Principle:

Principle #25Self-service

3Productivity

If memory pages are evenly distributed across partitions, then load balancing is achieved, but inter-partition traffic increases

Engineering Contradiction:

Improveload balancingVSAvoidinter-partition traffic

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system transitions from uniform data distribution to quality-aware data placement, where data pages are distributed based on their access patterns and affinity to specific processing units. Frequently accessed data pages are placed in the local memory of the processing units that access them most often, while less frequently accessed pages are distributed to other partitions. This non-uniform placement strategy reduces inter-partition traffic by keeping hot data local, while still maintaining overall load balance across the system.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system changes the distribution parameter from uniform allocation to access-pattern-based allocation. By using memory access profiles to determine data placement, the system dynamically adjusts where data pages are located based on actual usage patterns rather than following a fixed even distribution scheme. This parameter change allows the system to optimize for both load balancing and minimized inter-partition communication.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12141451B2Memory page access instrumentation

Publication Date: 2024.11.12 NVIDIA CORP

AI summary

Embodiments of the present disclosure relate to memory page access instrumentation for generating a memory access profile. The memory access profile may be used to co-locate data near the processing unit that accesses the data, reducing memory access energy by minimizing distances to access data that is co-located with a different processing unit (i.e., remote data). Execution thread arrays and memory pages for execution of a program are partitioned across multiple processing units. The partitions are then each mapped to a specific processing unit to minimize inter-partition traffic given the processing unit physical topology.