Multi-threaded Program Transformation for CPU Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CPUs are inefficient in executing CUDA programs due to limited concurrent thread execution and burdensome context switches required for synchronization barriers in SPMD applications, making direct mapping of CUDA threads to CPU threads unacceptably inefficient.
Innovation Solution
A method is introduced to transform a multi-threaded program by identifying kernel functions, enumerating barrier synchronization calls, replacing them with execution control commands, and inserting serialization control loops to enable efficient execution on a single-threaded processor core, effectively serializing the execution of parallel code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CUDA threads are directly mapped to CPU threads, then the program can be executed on a general purpose CPU, but the execution efficiency becomes unacceptably low due to limited concurrent thread execution and burdensome context switches
Solution Approach 1:
The invention segments the execution of CUDA kernel functions into multiple passes, with each pass handling a subset of threads. The kernel execution is divided into segments corresponding to different thread blocks, and each segment is executed sequentially on the CPU. This segmentation allows the system to maintain CUDA program compatibility while avoiding the overhead of concurrent thread management on the CPU.
Solution Approach 2:
The invention implements periodic action by repeatedly executing the same kernel function code with different thread block configurations. The CPU executes the kernel function multiple times in sequence, each time processing a different segment of threads, rather than attempting to execute all threads concurrently. This periodic re-execution pattern transforms the parallel execution model into a sequential one suitable for general purpose CPUs.
2Reliability
If synchronization barriers are implemented using conventional OS thread mapping, then thread synchronization can be achieved, but the overhead from context switches becomes burdensome
Solution Approach 1:
The invention extracts the synchronization barrier logic from the conventional OS thread context switch mechanism and implements it directly within the single-threaded CPU execution flow. By taking out the synchronization primitive handling from the OS thread layer and embedding it in the kernel execution layer, the system achieves synchronization without incurring context switch overhead.
Solution Approach 2:
The invention introduces an intermediary layer between the CUDA kernel code and the CPU execution engine. This intermediary translates CUDA synchronization primitives into equivalent single-threaded control flow constructs, mediating between the parallel execution model of CUDA and the sequential execution model of the CPU, thereby eliminating the need for OS context switches for synchronization.
3Adaptability or versatility
If a single-threaded processor core is used to execute multi-threaded programs, then compatibility with general purpose CPUs is achieved, but the inherent parallelism of the program cannot be utilized
Solution Approach 1:
The invention performs preliminary action by pre-processing the CUDA kernel function to identify synchronization barriers and segment the execution flow before actual execution. The kernel code is analyzed and prepared with embedded control logic that determines the execution segments and synchronization points in advance, allowing the single-threaded CPU to execute the program efficiently without losing the structural benefits of the original parallel design.
Data Source
AI summary
A technique is disclosed for executing a program designed for multi-threaded operation on a general purpose processor. Original source code for the program is transformed from a multi-threaded structure into a computationally equivalent single-threaded structure. A transform operation modifies the original source code to insert code constructs for serial thread execution. The transform operation also replaces synchronization barrier constructs in the original source code with synchronization barrier code that is configured to facilitate serialization. The transformed source code may then be conventionally compiled and advantageously executed on the general purpose processor.