GPU Matrix Multiplication Optimization Using Dynamic Programming

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The limited availability of graphics processing units (GPUs) and increased demand due to larger generative AI and large language models pose challenges in training and configuring these models efficiently.

Innovation Solution

A computing platform utilizing dynamic programming to optimize GPU operations by identifying optimal matrix multiplication orders through memoization and storing these orders in a lookup table, reducing the computational power required for matrix operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If GPUs are used to train generative AI models, then training capability is improved, but GPU availability deteriorates due to limited supply

Engineering Contradiction:

Improvetraining capabilityVSAvoidGPU availability

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent changes the operational parameters of GPU matrix multiplication by dynamically determining optimal multiplication orders based on matrix dimensions. This optimization reduces the number of operations required, making existing GPUs more efficient and effectively increasing available training capacity without requiring additional hardware.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system performs preliminary analysis of matrix dimension sets to identify and store optimal multiplication orders in a lookup table before actual training begins. This pre-computation approach allows GPUs to execute optimized operation sequences during training, improving productivity while reducing the need for additional GPU resources.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If larger generative AI models are developed, then model capability is improved, but GPU demand deteriorates due to increased resource requirements

Engineering Contradiction:

Improvemodel capabilityVSAvoidGPU demand

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent optimizes the operational parameters of matrix multiplication by dynamically determining optimal multiplication orders based on specific matrix dimensions. This reduces the computational operations required for each training iteration, allowing larger models to be trained with fewer GPUs than would traditionally be needed.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

Inventive Principle:

Principle #10Preliminary action

3Productivity

If dynamic programming is used to optimize matrix multiplication, then operation efficiency is improved, but computational overhead deteriorates during optimization

Engineering Contradiction:

Improveoperation efficiencyVSAvoidcomputational overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs the computationally intensive dynamic programming optimization in advance by analyzing matrix dimension sets and storing optimal multiplication orders in a lookup table. This preliminary action shifts the computational overhead to a pre-processing phase, allowing the actual training to benefit from optimized operation sequences without incurring repeated optimization overhead.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent creates a lookup table that copies and stores pre-computed optimal multiplication orders for various matrix dimension sets. During training, the system simply retrieves these pre-determined optimal sequences rather than performing dynamic programming optimization repeatedly, significantly reducing computational overhead while maintaining high operation efficiency.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20250328601A1Graphics processing unit (GPU) optimization using dynamic programming for generative artificial intelligence (AI) and large language models (LLM)

Publication Date: 2025.10.23 BANK OF AMERICA CORP

AI summary

A computing platform may receive matrix multiplication information indicating a plurality of matrix dimension sets for matrix multiplication. For each matrix dimension set, the computing platform may: 1) identify one or more multiplication variations, indicating different possible orders of operation for executing the corresponding multiplication, 2) perform memoization to identify, for each order of operation, a corresponding number of operations to complete the corresponding multiplication, 3) identify, based on the numbers of operations, a most efficient order of operation, and 4) store, in a lookup table, a relationship between the given matrix dimension set and the most efficient order of operation. The computing platform may identify matrix dimensions for a model configuration request, and identify, using the lookup table, a corresponding order of operations. The computing platform may iteratively train the generative AI model based on the order of operations, and deploy the generative AI model.