Multithreaded Processor Branch Instruction Merging

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

SIMT and SIMD processors face inefficiencies due to high branch operation costs, leading to idling and increased processing time, especially when dealing with branching operations in programs.

Innovation Solution

A data processing system and compiler that combine instructions from different branches to enable simultaneous execution on processing elements, allowing each element to execute multiple instructions per cycle, even during branching operations, using masking logic to optimize resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional SIMT/SIMD processing is used with branch operations, then program correctness is maintained, but processing elements idle and processing time increases

Engineering Contradiction:

Improveprogram correctnessVSAvoidprocessing throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent merges instructions from multiple branches into a single combined instruction that can be executed by all processing elements simultaneously. Instead of having different processing elements execute different branch instructions sequentially, the compiler combines compatible instructions from different branches into one unified instruction stream, allowing all elements to remain active and productive during branch execution.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The compiler performs preliminary analysis and combination of branch instructions during the compilation phase, before execution. By pre-processing the instruction stream to identify and merge compatible instructions from different branches, the system prepares an optimized execution plan that eliminates runtime idle periods without sacrificing program correctness.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If processing elements execute different branch instructions sequentially, then correct branch logic is maintained, but processing time increases due to idling

Engineering Contradiction:

Improvebranch logic correctnessVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system combines instructions from multiple branches into a single execution stream by identifying instructions that can be safely executed regardless of branch outcome. This merging process maintains correct branch logic through careful selection and ordering of instructions while eliminating the time loss associated with processing elements idling during branch execution.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The compiler transforms the instruction stream by changing the temporal parameters of instruction execution. Instead of sequential execution where elements wait for branch resolution, the system reorders and combines instructions to create a parallel execution model where multiple branch paths are processed simultaneously, reducing overall processing time while maintaining logical correctness.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If each processing element executes one instruction per cycle, then simplicity is maintained, but resource utilization decreases during branching

Engineering Contradiction:

Improveexecution simplicityVSAvoidresource utilization

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The combined instruction structure is designed to be universal, allowing the same instruction format to serve multiple branch paths simultaneously. This multi-functional instruction design enables processing elements to execute instructions from different branches through a unified mechanism, maintaining execution simplicity while improving resource utilization during branching operations.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The instruction stream is segmented into independent executable units that can be combined from different branches. By dividing the program into discrete, composable instruction segments, the system maintains the simplicity of individual instruction execution while enabling flexible combination of multiple branches, thereby improving resource utilization without increasing execution complexity.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11307862B2Compiling and combining instructions from different branches for execution in a processing element of a multithreaded processor

Publication Date: 2022.04.19 IMAGINATION TECH LTD

AI summary

A data processing system includes a processor operable to execute a program partitioned into a number of discrete instructions, the processor having multiple processing elements each capable of executing more than one instruction per cycle, and an interface configured to read a first program and, on detecting a branch operation by that program creating m number of branches each having a different sequence of instructions, combine an instruction from one of the branches with an instruction from at least one other branch so as to cause a processing element to execute the combined instructions during a single cycle.