Speed-Up Estimation for CPU to GPU Porting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Porting programs from CPUs to GPUs requires substantial restructuring and optimization, making it difficult to predict the performance advantage before significant effort and cost are incurred, as the speed-up benefit is only known after GPU code is written and optimized.

Innovation Solution

A method to estimate the speed-up of porting a program by measuring micro-architecture independent execution properties on a CPU and applying them to a model that predicts execution speed on a GPU, using a training set of programs optimized for both architectures and linear regression techniques.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If a program is ported from CPU to GPU, then execution speed is improved, but the effort and cost of porting increases substantially

Engineering Contradiction:

Improveexecution speedVSAvoidporting time

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent applies preliminary action by measuring execution properties and applying predictive models to the original CPU program before any GPU porting work begins. This allows the speed-up benefit to be estimated in advance, enabling stakeholders to make informed decisions about whether to proceed with porting, thereby avoiding wasted porting effort when speed-up would be insufficient.

Inventive Principle:

Principle #10Preliminary action

2Speed

If GPU code is written and optimized, then execution speed is improved, but the cost and effort of development increases

Engineering Contradiction:

Improveexecution speedVSAvoiddevelopment time

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent performs preliminary measurement and prediction on the CPU program before any GPU code is written. By using predictive models based on execution properties measured on the CPU, the system estimates the speed-up that would be achieved on GPU architecture, allowing development teams to avoid investing significant time in GPU porting when the expected benefit does not justify the effort.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If detailed instrumentation and micro-architecture understanding are used to measure performance, then measurement precision is improved, but device complexity and ease of operation worsen

Engineering Contradiction:

Improveperformance measurement accuracyVSAvoidease of program evaluation

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The patent employs a universal predictive model that can estimate GPU performance by measuring execution properties on any CPU program, regardless of the specific CPU micro-architecture. The model uses micro-architecture independent features such as instruction level parallelism, branch divergence, and special function utilization, which can be measured through automated analysis without requiring deep micro-architecture expertise or detailed instrumentation of the target GPU system.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS9384016B2Method of estimating program speed-up with highly parallel architectures

Publication Date: 2016.07.05 WISCONSIN ALUMNI RES FOUND

AI summary

The amount of speed-up that can be obtained by moving a program to a parallel architecture is determined by a model associating speed-up to micro-architecture independent features of the program execution. The model may be generated, for example, by linear regression, by evaluating programs that have been ported to parallel architectures where the micro-architecture independent features are known.