Near-Memory Data Casting for Reduced Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional computer architectures suffer from increased data transfer latency, energy consumption, and bandwidth consumption due to remote processing units being further away from memory, particularly in data-intensive applications like machine learning, leading to decreased performance.

Innovation Solution

Implementing a processing-in-memory component that offloads data casting operations such as upcast and downcast near the memory, allowing these operations to be performed closer to the memory, reducing data movement costs and improving data bus utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If data is transferred between remote processing units and memory, then processing can be performed, but data transfer latency increases and performance decreases

Engineering Contradiction:

Improvedata transfer speedVSAvoiddata transfer latency

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent introduces near-memory processing components as intermediaries between the remote processing units and the memory. These components include data casting logic that can perform data format conversions (upcast/downcast operations) locally at the memory, eliminating the need to transfer data back and forth between remote processing units and memory for formatting operations, thus reducing data transfer latency and improving speed.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If remote processing units are used, then processing tasks can be executed, but energy consumption and bandwidth requirements increase

Engineering Contradiction:

Improveprocessing capabilityVSAvoidenergy consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent implements data casting logic locally at the memory through near-memory processing components. This allows data format conversions (upcast and downcast operations) to be performed at the location where data is stored, rather than requiring remote processing units to fetch, process, and return data. This local processing approach maintains processing capability while significantly reducing energy consumption and bandwidth requirements by eliminating unnecessary data movements.

Inventive Principle:

Principle #3Local quality

3Ease of operation

If data casting operations are performed at remote processing units, then data can be processed, but data movement costs increase

Engineering Contradiction:

Improvedata processing capabilityVSAvoiddata movement costs

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The patent performs data casting operations (upcast and downcast) as preliminary actions at the near-memory processing components before data is transferred to remote processing units. By pre-converting data to the required formats at the memory location, the system eliminates the need for subsequent data movements and re-conversions at remote processing units, thereby reducing data movement costs while maintaining full data processing capability.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250110898A1Method and Apparatus for Collaborative Memory Accesses

Publication Date: 2025.04.03 ADVANCED MICRO DEVICES INC

AI summary

Method and apparatus for collaborative memory accesses is described. A system includes a memory controller that receives a command from a host. The command is associated with at least one of a plurality of data elements. The memory controller causes execution of data casting operations that adjust a bit size of the plurality of data elements to generate casted data elements. The system includes an interface for communicating data between the host and a memory.