Memory Port Replication for Latency Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Memory latency is a significant obstacle in computer systems, where over 90% of the time spent executing algorithmic functions is spent waiting for memory responses, leading to inefficient system performance, especially in systems with multiple instances operating in parallel.

Innovation Solution

Implementing multiple memory ports with a memory replication controller that directs read requests to the least-busy memory port by distributing data uniformly across all ports, ensuring that each memory port contains identical data and using address hashing units to convert virtual memory addresses to hardware addresses for optimal distribution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is stored uniquely across multiple memory ports, then total memory capacity increases, but read request distribution becomes non-uniform increasing latency

Engineering Contradiction:

Improvetotal memory capacityVSAvoidmemory read latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The memory system is segmented into multiple memory ports (first memory port, second memory port, etc.), each capable of storing unique data. This segmentation allows the system to increase total memory capacity while maintaining organized access structures through separate address hashing units for each port.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Address hashing units are introduced as intermediary components that translate virtual memory addresses to physical addresses and determine optimal memory port selection. These hashing units analyze read request patterns and distribute requests uniformly across multiple memory ports, preventing latency increases despite non-uniform data distribution.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If read requests are distributed non-uniformly across memory ports, then memory capacity utilization improves, but realized memory port read latency increases

Engineering Contradiction:

Improvememory capacity utilizationVSAvoidmemory port read latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system dynamically adjusts read request distribution across memory ports based on real-time conditions. Address hashing units continuously analyze access patterns and modify distribution strategies to maintain uniform request allocation, ensuring low latency while adapting to changing workload demands.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The address hashing units modify address translation parameters and distribution algorithms based on observed access patterns. By changing how virtual addresses map to physical ports dynamically, the system maintains uniform request distribution across memory ports, preventing latency increases while utilizing full memory capacity.

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If multiple memory ports are used with unique data storage, then system memory capacity increases, but access latency becomes dependent on other units' read frequency

Engineering Contradiction:

Improvesystem memory capacityVSAvoidaccess latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

Each memory port is designed with universal functionality to handle any type of data access independently. The address hashing units provide multi-functional address translation capabilities that work across all memory ports uniformly, ensuring that access latency remains consistent regardless of which port is accessed or what data is being retrieved.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Address hashing units serve as intermediary components that decouple the dependency between parallel algorithm instances and memory ports. By introducing this abstraction layer that uniformly distributes requests regardless of port workload, the system eliminates latency dependence on other units' read frequencies while maintaining expanded memory capacity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS8850101B2System and method to reduce memory access latencies using selective replication across multiple memory ports

Publication Date: 2014.09.30 MARVELL ASIA PTE LTD

AI summary

In one embodiment, a system comprises a plurality of memory ports. The memory ports are distributed into a plurality of subsets, where each subset is identified by a subset index. The system further comprises a first address hashing unit configured to receive a request including at least one virtual memory address. Each virtual memory address is associated with a replication factor, and the virtual memory address refers to graph data. The first address hashing unit translates the replication factor into a corresponding subset index based on the virtual memory address, and converts the virtual memory address to a hardware based memory address. The hardware based address refers to data in the memory ports within a subset indicated by the corresponding subset index.