Bandwidth-Aware Multithreading With Banked Front-End Structures

The front-end pipeline stages are segmented into multiple independent paths, allowing different threads to be processed simultaneously in different segments. This segmentation enables multi-threaded access to front-end resources without requiring complex thread ownership management, as each thread can progress through dedicated pipeline segments in parallel.

The patent transitions from a single-dimension thread execution model (one thread per pipeline stage per cycle) to a multi-dimensional model where multiple threads can occupy different pipeline stages simultaneously. This dimensional expansion allows front-end structures to service multiple threads in the same clock cycle, effectively increasing bandwidth utilization without proportionally increasing complexity.

Front-end structures such as branch prediction units and instruction caches are divided into multiple independent segments or banks. Each thread can access different segments simultaneously without interfering with other threads, eliminating resource contention while maintaining high bandwidth utilization. For example, the branch prediction table is segmented into multiple banks that can be accessed in parallel by different threads.

The patent introduces intermediary structures such as separate instruction queues and decode buffers for each thread, which act as buffers between the front-end structures and the execution units. These intermediaries prevent direct conflicts between threads accessing shared resources, allowing simultaneous access without contention by mediating resource allocation.

The front-end structures are divided into banks that are replicated for each thread. While this increases the total number of structures, each bank is a simplified replica of the original structure, allowing parallel access without requiring complex arbitration logic. The segmentation into banks enables straightforward address decoding where the thread ID directly selects the appropriate bank.

The patent implements banking of front-end structures even when full utilization may not always be achieved, as the overhead of the banking structure is justified by the potential for doubled bandwidth utilization. The excessive action of providing redundant structures for each thread ensures that when both threads are active, full bandwidth is available, and the structure complexity is a worthwhile trade-off for the performance gain.