Memory Access Processor Segmentation for Power Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current computing systems face inefficiencies in power consumption and memory access latency due to the 'Memory Wall' problem, where processors are stalled waiting for memory data, leading to wasted memory bandwidth and power usage, especially when only partial data is needed.

Innovation Solution

A computing memory system with an execution unit and an access processor that are logically separated, where the access processor reads data from the memory system, executes it, and reserves time-slots for writing results back into the memory system based on execution times, optimizing data transfer and reducing power consumption by integrating processing capabilities within the memory system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the processor uses traditional memory access mechanisms to overcome the Memory Wall, then memory access latency is reduced and execution unit utilization is maximized, but power consumption increases and memory bandwidth is wasted

Engineering Contradiction:

Improveexecution unit utilizationVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system segments the memory access function by introducing a dedicated access processor that handles all memory operations separately from the execution units. This segmentation allows the execution units to focus solely on computation while the access processor manages memory bandwidth and latency, resolving the contradiction between maximizing execution utilization and reducing power consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The access processor acts as an intermediary between the execution units and the memory system. It receives data requests from execution units, manages the memory access protocol, and delivers data back to the execution units. This intermediary layer optimizes memory access patterns and reduces unnecessary bandwidth consumption, thereby reducing overall power consumption while maintaining high execution unit utilization.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If the processor retrieves complete cache lines from memory system, then data availability is ensured, but memory bandwidth is wasted when only partial data is needed

Engineering Contradiction:

Improvedata availabilityVSAvoidmemory bandwidth waste

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The access processor implements partial data retrieval by fetching only the specific bytes or words needed from memory rather than complete cache lines. This partial action approach ensures data availability for the execution units while significantly reducing memory bandwidth consumption and the associated power waste, directly addressing the contradiction between reliability and energy loss.

Inventive Principle:

Principle #16Partial or excessive action

3Loss of time

If the processor implements prefetching mechanisms to reduce access latency, then average access latency is reduced, but power consumption increases and memory bandwidth is wasted on partially processed data

Engineering Contradiction:

Improvememory access latencyVSAvoidpower consumption

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The access processor performs preliminary actions by pre-fetching data into dedicated buffers before the execution units actually need it. This allows the execution units to continue processing without waiting for memory access, reducing latency. The preliminary action is optimized to fetch only necessary data, avoiding the power waste associated with traditional prefetching mechanisms.

Inventive Principle:

Principle #10Preliminary action

4Speed

If the system integrates processing capabilities within the memory system, then data transfer distance is minimized and processing speed increases, but device complexity increases

Engineering Contradiction:

Improveprocessing speedVSAvoidsystem architecture complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The system merges the access processor with the memory system, integrating data storage and data access functions into a single unified architecture. This merging minimizes data transfer distance between memory and processing units, significantly increasing processing speed. The integration is designed to manage complexity through functional consolidation rather than adding separate complex components.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS9959202B2Memory and processor hierarchy to improve power efficiency

Publication Date: 2018.05.01 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A computing memory includes an execution unit and an access processor coupled with a memory system, where the execution unit and the access processor are logically separated units. The execution unit is for processing operand data. The access processor is for providing operand data and configuration data to the execution unit. The access processor reads operand data from the memory system and sends the operand data to the execution unit. The execution unit executes the operand data according to the provided configuration data. The access processor includes information about execution times of operations of the execution unit for the provided configuration. The access processor reserves time-slots for writing execution unit results provided by the execution unit into selected locations in the memory system based on the information about the execution times, upon sending at least one of the operand data and the configuration data to the execution unit.