LLM SoC Memory Architecture for Lower-Power DDR Data Paths

Solution Overview

Problem

Current hardware architectures for large language model (LLM)-based applications face significant power consumption and thermal dissipation challenges due to high-speed memory usage, leading to thermal throttling and performance degradation.

Innovation Solution

An improved SoC architecture integrates LLM processing blocks with memory controllers and DDR PHY interfaces, reducing data paths and operating within the DDR PHY voltage domain to enhance power efficiency and thermal management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If high-speed memory interfaces are used for LLM processing, then data transfer speed is improved, but power consumption and thermal dissipation increase

Engineering Contradiction:

Improvedata transfer speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent merges the LLM processing block with the memory controller and PHY interface into a single integrated unit. This consolidation eliminates separate high-speed interfaces between these components, reducing the overall data path length and the number of high-power interface circuits needed, thereby achieving power savings while maintaining data transfer speed.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent changes the architectural dimension by integrating the LLM processing block directly within the memory controller's voltage domain and physical location. This spatial reorganization allows data to be processed closer to its source, reducing the need for long-distance high-speed signal transmission and associated power consumption.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Speed

If high-speed memory interfaces are used for LLM processing, then data transfer speed is improved, but thermal dissipation increases

Engineering Contradiction:

Improvedata transfer speedVSAvoidthermal dissipation

Core Design Contradiction:

SpeedVSTemperature

Solution Approach 1:

The patent merges the LLM processing block with the memory controller and PHY interface into a single integrated unit. This consolidation eliminates separate high-speed interfaces between these components, reducing the overall data path length and the number of high-power interface circuits needed, thereby achieving power savings while maintaining data transfer speed.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent converts the potential harm of thermal dissipation by operating the LLM processing block within the DDR PHY voltage domain, which allows for lower operating voltages. This voltage domain selection inherently reduces power consumption and thermal generation while still enabling high-speed operation through efficient data path integration.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

3Use of energy by moving object

If LLM processing block is integrated with memory controller and PHY interface, then power consumption is reduced, but device complexity increases

Engineering Contradiction:

Improvepower consumptionVSAvoidarchitecture complexity

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The patent merges the LLM processing block with the memory controller and PHY interface into a single integrated unit. This consolidation eliminates separate high-speed interfaces between these components, reducing the overall data path length and the number of high-power interface circuits needed, thereby achieving power savings while maintaining data transfer speed.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12505287B2Power optimized architecture for large language model-based applications

Publication Date: 2025.12.23 QUALCOMM INC

AI summary

A large language model (LLM)-based machine learning (ML) system-on-chip (SoC) is described. The LLM-based SoC includes a first LLM processing block. The LLM-based SoC also includes a first memory controller coupled to the first LLM processing block. The LLM-based SoC further includes a first physical layer (PHY) memory interface coupled to the first LLM processing block through the first memory controller. The LLM-based SoC further includes a memory coupled to the first PHY memory interface as a first data channel.