Memory Controller Programmable Atomic Operations

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current memory controllers are not optimized for high-performance, energy-efficient operations in compute-intensive applications like machine learning, AI, and 5G networking, which require frequent and low-latency memory accesses, and lack support for programmable atomic operations.

Innovation Solution

A memory controller design that provides high performance and energy efficiency by supporting both predetermined and programmable atomic operations, with optimized throughput and latency, using programmable atomic operations circuitry that allows user-defined operations and direct data paths to minimize network congestion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional memory controllers are used, then general-purpose functionality is maintained, but performance and energy efficiency in compute-intensive applications are insufficient

Engineering Contradiction:
Improvememory access throughputVSAvoidapplication-specific optimization
Core Design Contradiction:
ProductivityVSAdaptability or versatility

Solution Approach 1:

The memory controller implements different quality levels of service for different memory access patterns. It provides highly optimized performance for atomic operations and frequently accessed data, while maintaining acceptable performance for other operations. This local optimization resolves the contradiction by concentrating resources where they are most needed without completely sacrificing general-purpose functionality.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The memory controller dynamically adjusts its behavior based on the type of memory access being performed. It prioritizes atomic operations and adapts its arbitration logic in real-time based on workload characteristics. This dynamic adaptation allows the system to achieve high performance for compute-intensive applications while maintaining versatility across different workloads.

Inventive Principle:
Principle #15Dynamics

2Productivity

If frequent memory accesses are performed for compute-intensive operations, then productivity improves, but latency increases

Engineering Contradiction:
Improvecompute-intensive operation throughputVSAvoidmemory access latency
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The memory controller performs preliminary actions by pre-fetching data that will be needed for upcoming compute-intensive operations. It anticipates memory access patterns and loads data into buffers before they are actually needed, reducing latency during the critical computation phases. This is particularly effective for operations like FFT and FIR filtering where data access patterns are predictable.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The memory controller maintains continuous useful action by keeping data in buffers and caches between memory accesses. Instead of allowing idle time between memory operations, it maintains ready-to-use data structures that can be immediately accessed, ensuring continuous productive operation and eliminating latency gaps in the compute-intensive workload.

Inventive Principle:
Principle #20Continuity of useful action

3Reliability

If atomic operations are performed, then data consistency is maintained, but throughput is reduced compared to non-atomic operations

Engineering Contradiction:
Improvedata consistencyVSAvoidmemory access throughput
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The memory controller segments atomic operations into distinct phases: read phase, compute phase, and write phase. By separating these operations and using dedicated buffers for each phase, it allows overlapping of operations and eliminates the traditional sequential bottleneck. This segmentation enables atomic operations to achieve throughput comparable to non-atomic operations while maintaining data consistency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The memory controller introduces intermediary buffers between the processor and main memory for atomic operations. These buffers act as mediators that decouple the read-modify-write sequence, allowing the buffer to hold intermediate values and enabling parallel execution of atomic operations. This intermediary structure resolves the throughput penalty by providing a high-speed buffer zone that eliminates repeated main memory accesses.

Inventive Principle:
Principle #24Intermediary (Mediator)

4Ease of manufacture

If predetermined atomic operations are supported, then hardware implementation is simplified, but programmable atomic operations cannot be performed

Engineering Contradiction:
Improvehardware implementation simplicityVSAvoidprogrammable operation support
Core Design Contradiction:
Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The memory controller implements a universal atomic operation interface that can handle both predetermined and programmable operations through a single unified architecture. The same buffer infrastructure and arbitration logic serve both fixed and user-defined atomic operations, eliminating the need for separate hardware paths. This multi-functionality resolves the contradiction by showing that programmable operations can be achieved without sacrificing hardware simplicity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The memory controller uses parameter changes to distinguish between predetermined and programmable atomic operations. Instead of different hardware paths, it changes control parameters and instruction formats to route different operation types through the same infrastructure. This parameter-based differentiation maintains hardware simplicity while enabling programmable operations through software-configurable behavior.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS12019920B2Memory controller with programmable atomic operations
Publication Date: 2024.06.25 MICRON TECHNOLOGY INC
  • US12019920B2 patent drawing
  • US12019920B2 patent drawing
  • US12019920B2 patent drawing

AI summary

A memory controller circuit is disclosed which is coupleable to a first memory circuit, such as DRAM, and includes: a first memory control circuit to read from or write to the first memory circuit; a second memory circuit, such as SRAM; a second memory control circuit adapted to read from the second memory circuit in response to a read request when the requested data is stored in the second memory circuit, and otherwise to transfer the read request to the first memory control circuit; predetermined atomic operations circuitry; and programmable atomic operations circuitry adapted to perform at least one programmable atomic operation. The second memory control circuit also transfers a received programmable atomic operation request to the programmable atomic operations circuitry and sets a hazard bit for a cache line of the second memory circuit.