Time-critical multithreaded processor

Abstract

A multithreaded processor architecture for time-critical applications featuring a specialized scheduler and resource management unit (RMU). The architecture employs dynamic thread execution modes including deterministic execution with dedicated resource allocation, progress-monitored execution with worst case execution time (WCET) guarantees, and best-effort execution for non-critical tasks. The time-critical scheduler guarantees deterministic performance through fixed execution slot allocation and progress tracking with dynamic priority adjustment. The RMU enhances this capability by managing shared resources including I / O devices, buses, interconnects, and memory hierarchies through resource partitioning, access control, quality of service management, and monitoring. This integrated approach enables efficient resource utilization while maintaining strict timing guarantees for critical threads, allowing time-critical and best-effort threads to operate concurrently without interference.

Description

[0001] TIME-CRITICAL MULTITHREADED PROCESSOR

[0002] Field of the Invention

[0003] The present invention relates to the field of processor microarchitecture, and more specifically to an apparatus and method able to execute instructions in time-critical applications across multiple threads simultaneously.

[0004] Background

[0005] While conventional processors that run a single thread at a time can be operated such that interference from other processors is negligible in order to guarantee deterministic behavior, or at least guarantee a worst case execution time (WCET), such use can lead to complex system design and can be an inefficient use of resources. Alternatively, conventional multithreaded processors typically schedule threads fairly based on availability of resources so determinism and WCET is not guaranteed.

[0006] Simultaneous Multithreading (SMT) is a processor design technique in which multiple independent threads issue instructions to different execution units within a single processor cycle. SMT is an evolution of superscalar architectures that improves processor utilization by exploiting thread-level parallelism alongside instruction-level parallelism.

[0007] In conventional SMT implementations, a single physical processor appears as multiple logical processors to the operating system. The processor maintains separate architectural states (register files, program counters, etc.) for each thread while sharing the physical execution resources such as caches, execution units, and branch predictors. This approach allows instructions from different threads to be executed simultaneously on shared functional units, filling pipeline bubbles that would otherwise occur due to dependencies, cache misses, or branch mispredictions within a single thread. Implementations of SMT include Intel's Hyper-Threading Technology, IBM's POWER architecture and AMD’s Zen microarchitectures.

[0008] One challenge in SMT design is sharing resource allocation between threads. Scheduling of ready threads to functional units, also called instruction issue or dispatch, determines which instructions from which threads get access to execution resources in each cycle. These scheduling decisions typically occur at the core of the processor pipeline, after instructions have been fetched and decoded but before they enter execution units. The scheduler must consider not just the availability of functional units, but also dependencies between instructions, the status of the reorder buffer, and the availability of physical registers.

[0009] Round-robin scheduling is a simple approach where the processor alternates between threads in a fixed pattern, giving each equal opportunity to issue instructions. While straightforward to implement, this approach doesn't account for the varying resource needs of different threads or adapt to changing execution conditions.

[0010] Priority-based scheduling can assign priorities to threads based on factors such as: thread stall frequency (prioritizing threads that stall less frequently), resource usage history (balancing resource allocation), thread criticality (based on OS-level hints or microarchitectural indicators) and / or long-latency operation detection (deprioritizing threads waiting on memory).

[0011] Speculative instruction selection with dynamic resource partitioning allows the processor to predict which instructions are likely to make the most progress and allocates functional units accordingly.

[0012] Typical instruction scheduling policies balance several competing objectives: maximizing throughput across all threads, ensuring fairness (preventing thread starvation), minimizing the impact of negative thread interactions (such as cache pollution) and adapting to changing workload characteristics. However, typical systems don’t guarantee deterministic execution or even quantifiable measurable progress for an individual thread so are not well suited for execution of time-critical applications.

[0013] Description of the Figures

[0014] Fig. 1 illustrates elements of the processor core in an embodiment of the invention.

[0015] Fig. 2 illustrates a time vs. progress graph.

[0016] Fig. 3 illustrates a time vs. progress graph.

[0017] Fig. 4 illustrates a time vs. progress graph.

[0018] Fig. 5 illustrates a time vs. progress graph. Detailed Description

[0019] Embodiments of the present invention can exploit the advantages of a multithreaded processor design (efficiency of resource utilization and lower system complexity) while also managing threads where determinism or WCET is needed.

[0020] Specifically, individual threads can be fully deterministic such that they are isolated and do not suffer from interference from other threads, or alternatively can have variability in execution time but can be guaranteed to complete within a specific time. This can be accomplished through the use of dedicated resource allocations including bus resources, buffers, cache regions and guaranteed execution unit slots. Furthermore, dynamic thread priority can be used to implement an asymmetric scheduler and asymmetric access to resources. In this case, thread progress can be monitored such that WCET can be guaranteed by dynamically altering the priority and resources allocated to a thread.

[0021] In some implementations, threads can operate in three modes:

[0022] 1. Deterministic mode: the thread is completely isolated from other threads. The use of private resource allocations such as scratch pad memory and fixed execution slots can be used to guarantee deterministic execution progress. The cycle by cycle progress of a deterministic thread is fixed and guaranteed. Lockstep operation can be used with two threads for critical functions.

[0023] 2. Progress monitored (WCET) mode: threads in this mode are not isolated from each other from the standpoint of memory or execution resources, but have their individual progress monitored with dynamically adjusted priority. The progress monitoring mechanism is used to dynamically modify the thread behavior when the rules associated with the monitoring mechanism fires. The system guarantees that enough performance exists to complete a task at a given deadline if the thread is promoted to maximum priority, or converted to deterministic mode.

[0024] 3. Best effort mode: threads in this mode are not isolated and are not monitored for progress. Dynamic priorities can still be used. Threads in this mode are used for non-time critical and background or housekeeping functions. An advantage of having best effort threads running on the same engine is increase efficiency and lower complexity. The proposed architecture is an asymmetric multithreaded architecture where the resources can be assigned to the threads in different ways dynamically with different characteristics.

[0025] Referring to Fig. 1, the core data pipeline of a processor consists of physical register files 104 and functional units 107. In each cycle operand values are read from the physical register files 104 and operated on by functional units 104 and written back to the physical register files 104. In the case that a value being computed will be consumed in the very next cycle, a bypass path is provided so that back to back ALU operations are possible. In the case of load / store memory operations, memory system 108 receives operations and for loads the results are written back to the physical register file 104 and / or use the bypass path.

[0026] This core data pipeline takes place in an out of order execution machine after instruction fetch and register renaming. Time-critical SMT scheduler 103 is responsible for determining on a cycle by cycle basis which instructions from which threads get executed by which functional units. The time-critical SMT scheduler receives input from the Instruction Queues / Reorder Buffer (ROB) 101. For each of the active threads, the Instruction Queues / ROB 101 keeps track of which instructions are ready for execution and maintains architectural state for each active thread.

[0027] The time-critical SMT scheduler 103 receives input from Policy Engine 102, which can be dynamically configured to determine which threads have access to which functional units in which cycles. In certain embodiments the time-critical SMT scheduler 103 also can receive input from Thread Progress Tracker 105, which maintains information for each active thread the progress it is making compared to a target progress. This information can be used to dynamically adjust thread priority such that a guaranteed worst case execution time (WCET) is maintained.

[0028] Resource Management Unit (RMU)

[0029] One aspect of the present invention is a Resource Management Unit (RMU). Conventional processors utilize Memory Management Units (MMUs) as a way to manage memory resources efficiently. An MMU is a hardware component responsible for translating virtual addresses to physical addresses in a computer system. Its major functions include: Address Translation (converting virtual addresses used by programs into physical addresses in RAM); Memory Protection (isolating memory spaces between processes to prevent unauthorized access); Paging and Segmentation (supporting memory virtualization by managing pages and memory segments); Cache Control (assisting in managing memory caching strategies and ensuring cache coherency); and Access Rights Checking (enforcing read / write / execute permissions for each memory region or page).

[0030] Extending the idea of the MMU is a new hardware concept: the Resource Management Unit (RMU). In today’s complex architectures, a wide range of system resources are shared dynamically between tasks, threads, and cores, including: I / O Interfaces; Buses and Interconnects; Communication Queues; Caches (instruct! on / data caches, shared caches); Memory subsystems (shared RAMs, scratchpads); Exception and Interrupt Controllers; and Hardware Accelerators (GPUs, DSPs, NPUs, TPUs, etc.). The sharing of these resources is a major source of response time variability, unpredictability, and potential contention, especially in multicore and real-time systems.

[0031] The RMU is a specialized hardware structure that aims to assign, protect, and control the use of these shared resources. In particular the RMU offers: Resource Partitioning (assigning specific resources to specific tasks or cores to guarantee availability and avoid interference); Access Control (enforcing access rights and priorities for each resource); Reservation and Preemption Policies (supporting dynamic reservation or preemption of resources under system policies); Monitoring and Accounting (tracking usage patterns and detect contention or violations in real time); Quality of Service (QoS) Management (enabling predictable latency and throughput guarantees for critical workloads); and Fault Isolation (containing errors caused by resource misuse and support safe recovery). In essence, the RMU plays a role for general resources similar to what the MMU does for memory: turning raw shared hardware into a controlled, protected, and virtualized resource environment.

[0032] In a virtualized system — where multiple Virtual Machines (VMs) or containers share the same physical hardware — the RMU becomes even more critical. Virtual Resource Views: The RMU creates a virtualized view of hardware resources for each VM or container, mapping logical resource requests to physical resources securely and transparently. Per-VM Resource Allocation: Resources can be allocated or restricted per VM to ensure strong isolation and predictable performance. Dynamic Reallocation: Resources can be reassigned dynamically between VMs as workloads change, based on hypervisor or RMU-controller policies. Secure Isolation: The RMU enforces hardware-based barriers between VMs, preventing resource contention, side-channel attacks, and denial-of-service scenarios. QoS per VM: Critical VMs can receive guaranteed access and service levels without interference from other VMs. Hypervisor Integration: Hypervisors interact with the RMU to request partitions, monitor usage, and enforce protection policies. Virtual Interrupt and Exception Management: RMU can virtualize and route interrupts and exceptions precisely, enhancing performance and security for multi-tenant environments.

[0033] Modern computing trends move beyond traditional servers into disaggregated architectures, where compute, memory, storage, and accelerators are physically separated but dynamically composed over high-speed fabrics (e.g., CXL, Ethernet, and PCIe). In these environments, the RMU plays a key role in the following areas. Global Resource Mapping: The RMU manages not only local resources but also remote resources across a fabric, creating a unified, secure resource map for VMs or services. Remote Resource Control: It can enforce access control and protection even for resources physically located on different devices or racks. Dynamic Composition: RMUs enable on-demand allocation and reallocation of remote memory, accelerators, or storage to VMs or applications, supporting elastic scaling. Fabric QoS Enforcement: It monitors and ensures that disaggregated communication channels (e.g., CXL links, network paths) meet required latency and bandwidth guarantees. Isolation Across Domains: RMUs help maintain strong security and isolation even when resources are shared across multiple tenants, servers, or data centers. Support for Stateless Systems: RMUs allow resources to be detached or replaced without disrupting the virtualized environment, enabling better resilience and flexibility. In disaggregated virtualization, the RMU acts as a distributed hardware manager, orchestrating secure, efficient, and predictable use of an ever-changing pool of heterogeneous hardware resources.

[0034] In real-time and deterministic systems, where the timely and predictable execution of tasks is paramount, the RMU plays a crucial role in ensuring that shared resources do not compromise system performance or timing constraints. These systems are found in safety-critical and high- performance applications such as autonomous vehicles, industrial automation, aerospace, and telecommunications. Key benefits of RMUs in these environments include the following ways. Deterministic Resource Allocation: RMUs guarantee that resources (e.g., CPUs, accelerators, and I / O devices) are allocated with strict priorities and without resource contention, ensuring tasks meet their deadlines. Hard Isolation of Resources: RMUs isolate resources such as caches, and memory so that critical real-time tasks are never disrupted by lower-priority tasks, avoiding worstcase execution time (WCET) violations. Guaranteed Latency: With RMUs, system architects can design and enforce strict latency guarantees for critical tasks by managing how and when resources are accessed, ensuring bounded latencies even in complex multi-tasking environments. Resource Reservation: RMUs support reserving exclusive access to certain hardware resources for real-time tasks, ensuring they are not preempted by non-critical workloads. Time-Sensitive QoS: The RMU allows for time-sensitive Quality of Service (QoS) policies, ensuring that critical applications are provided with enough bandwidth, processing power, and memory without interference. Predictability with Resource Sharing: Despite the sharing of hardware resources across tasks or VMs, RMUs enable systems to function predictably and deterministically without compromising performance, by preventing resource contention and enforcing strict execution policies. In this way, the RMU enables safe and efficient sharing of resources in systems where timing and reliability are non-negotiable, ensuring that critical applications can operate within their strict timing constraints while still benefiting from the efficiencies of resource sharing.

[0035] In one embodiment, the RMU is integrated into a system with processor cores (multiple execution units capable of simultaneous or parallel task execution) and shared resources (including buses, I / O, accelerators, shared caches, and main memory). The RMU is positioned between the processor cores and the shared resources and comprises: a Resource Mapping Unit (maps logical resource identifiers from each task or virtual machine to the physical counterparts), an Access Control Engine (enforces isolation policies and access rights), a QoS Policy Module (maintains performance guarantees by regulating access bandwidth or latency), a Monitoring and Accounting Engine (tracks usage patterns and logs policy violations or overuse), a Virtualization Interface (interacts with hypervisors or orchestration software to provision resources), a Communication Fabric (in disaggregated variants, connects external memory, accelerators, or storage via CXL / PCIe / Ethernet) and an Interrupt and Exception Router (routes and virtualizes interrupts per execution context).

[0036] Deterministic Thread Execution

[0037] Turning to scheduling of deterministic threads, one embodiment of the present invention involves enhancing SMT systems with dedicated scheduling slots to provide deterministic performance guarantees for time-critical threads. In this approach, the throughput benefits of SMT are combined with quality-of-service guarantees important for real-time systems.

[0038] In order to guarantee that a thread makes progress at a fixed rate a reservation-based scheduling mechanism can be implemented where each thread is allocated a specific number of issue slots within a defined time window. The hardware implements a slot reservation table that tracks the allocation of execution resources over time. Each thread is assigned a minimum number of scheduling slots per scheduling period (for example, 20 slots per 100 cycles). These reservations can be maintained by a progress tracking unit that ensures threads receive their guaranteed allocation.

[0039] A slot reservation system can provide deterministic guarantees by allocating dedicated execution resources to real-time threads:

[0040] 1. Temporal Partitioning: The processor cycle timeline is divided into fixed-length epochs (e.g., 1000 cycles), with a guaranteed minimum number of instruction slots allocated to each realtime thread within each epoch.

[0041] 2. Resource Reservation: Beyond execution slots, the system can reserve portions of shared resources like cache ways, memory bandwidth, and reorder buffer entries exclusively for real-time threads.

[0042] 3. Admission Control: The system performs an analysis when a new real-time thread enters, calculating whether sufficient slots exist to guarantee its requirements without violating existing guarantees.

[0043] 4. Non-interference Guarantees: By strictly isolating resource usage between threads, the system ensures that non-real-time threads cannot delay operations of real-time threads.

[0044] Achieving truly deterministic execution time in SMT architectures represents an even stricter requirement than WCET guarantees. Deterministic execution means that a thread will complete in exactly the same number of cycles every time it runs, regardless of what other threads are doing. This can be crucial for applications like safety-critical systems or security applications where timing predictability is essential.

[0045] 1. Strict Temporal Partitioning: The processor implements rigid time-division multiplexing where each thread receives exactly the same instruction slots in each scheduling period. Rather than just guaranteeing a minimum, the system allocates precisely the same number of slots in the same pattern during each period.

[0046] 2. Complete Resource Isolation: Beyond execution slots, all shared resources that could cause timing variations are strictly partitioned, for example dedicated cache ways for each thread; reserved memory bandwidth channels; partitioned branch predictors and other speculative structures; and isolated TLB entries.

[0047] 3. Progress-Based Throttling: If a thread attempts to progress faster than its deterministic rate (perhaps due to favorable branch predictions or cache hits), the system throttles it to maintain the predetermined execution rate.

[0048] 4. Deficit Compensation with Exact Correction: When monitoring reveals that a thread has fallen behind its expected progress due to unavoidable events (like hardware interrupts), the dynamic priority system allocates exactly enough additional slots to bring the thread back to its predetermined execution schedule.

[0049] 5. Elimination of Timing Channels: The system removes sources of non-determinism such as: variable-latency instructions (using worst-case timing for all operations); DVFS and thermal throttling effects (fixing processor frequency); and interference from system management interrupts.

[0050] In certain cases, the processor can: assign a deterministic thread to a fixed set of execution units, reserve fixed portions of the LI and L2 caches, guarantee a fixed number of memory requests per cycle and disable speculative execution features that could cause timing variations.

[0051] When the deterministic thread begins execution, the hardware establishes a deterministic progress curve — essentially a function mapping cycle count to expected instruction retirement count. The progress monitoring hardware continuously compares actual progress to this curve, making corrections as needed.

[0052] If external events cause deviations (like unavoidable interrupts), the system compensates with exactly enough additional slots to return to the deterministic timeline.

[0053] Progress Guaranteed Threads

[0054] An alternative approach to deterministic threads is to have a more flexible slot assignment and use dynamic priority to guarantee a worst case execution time (WCET) to support time-critical threads. In this case, when a time-critical thread has not met its progress quota, the scheduler prioritizes its instructions higher even if other threads might make better use of the resources. This prioritization could be implemented using a token bucket algorithm, where each thread accumulates tokens at its guaranteed rate and spends them when issuing instructions. In some embodiments the implementation involves thread progress monitoring (hardware counters to track committed instructions per thread), dynamic priority logic (circuit that controls scheduling decisions to enforce progress guarantees), admission control (mechanism to determine if the system can accommodate a new thread's guarantee without violating existing commitments), and resource partitioning (static partitioning of some resources (e.g. reorder buffer entries) to prevent blocking).

[0055] This approach is particularly valuable for mixed- workload environments where some threads have real-time constraints while others are throughput-oriented. For example, in a system running both safety critical threads and background computation, the safety critical threads could maintain performance guarantees regardless of background load.

[0056] Thread progress monitoring coupled with dynamic prioritization creates a self-correcting system where threads that fall behind their expected progress automatically receive additional resources to catch up.

[0057] Referring to Figures 2, 3, 4 and 5, graphs of time vs. progress are shown. Fig. 2 illustrates an empty graph with a dotted line showing the average completion rate to meet the target completion time. Execution of critical tasks must be close to the line to be completed in time. When under the line, more resources must be allocated to speed up the execution time. Fig. 3 illustrates the beginning of two tasks (one critical task and one best-effort task) and their progress monitoring through time step 3. As illustrated, an expected completion time can be computed at each stage and the projected completion time has improved for the critical task but it is still under the target, so increased priority is necessary. Fig. 4 illustrates the two tasks with progress monitoring through time step 8. As shown, the system performed an update at time step 4 that allocated more resources to the critical task which is now expected to complete in under the expected completion time. Fig. 5 illustrates the two tasks at completion (time step 14) with the critical task completing at the expected completion time.

[0058] The core of this system is a hardware-based progress deficit tracker that continuously monitors each thread's actual progress against its expected progress target. In one embodiment, for each active thread, the processor maintains: a progress counter tracking committed instructions or other meaningful progress metrics, an expected progress register indicating where the thread should be at the current time, and a deficit / surplus register calculating the difference between actual and expected progress.

[0059] At regular intervals (e.g., every 1000 cycles), the hardware compares actual progress to expected progress and update the deficit / surplus value. The expected completion time can be computed based on the current progress. When a thread's deficit exceeds certain thresholds, the priority adjustment mechanism engages in several ways:

[0060] 1. Proportional compensation: The priority boost could be proportional to the deficit, creating a system similar to a PID controller where larger deficits result in more aggressive compensation.

[0061] 2. Age-based amplification: Deficits that persist over time could be weighted more heavily, preventing situations where a thread falls permanently behind.

[0062] 3. Resource allocation adjustments: Beyond instruction scheduling, the system could also adjust cache partitioning, memory bandwidth allocation, and other shared resources to favor threads with significant deficits.

[0063] 4. Preemptive scheduling: In extreme cases, the system can temporarily suspend low- priority threads entirely to allow high-deficit threads to catch up quickly.

[0064] A variety of progress metrics can be used in different embodiments. For example, different thread types might require different progress metrics - instructions retired might be appropriate for compute-bound threads, while I / O operations completed might be more meaningful for I / O-bound threads. Also, the algorithm can be adjusted to prevent oscillation. Damping factors and hysteresis can be used to smooth transitions.

[0065] By creating this feedback loop between observed progress and scheduling priority, the system naturally adapts to varying workload characteristics and interference patterns, maintaining performance guarantees even under challenging conditions.

[0066] These approaches enable SMT architectures to support mixed-criticality workloads where realtime threads with strict WCET requirements can safely coexist with best-effort threads focused on throughput, making SMT viable for a wider range of applications including safety- critical systems.

[0067] In many applications there would be deterministic and WCET for a subset of time-critical threads while allowing conventional SMT operation for others, creating a system that can support both timing-critical and throughput-oriented workloads simultaneously. In this way, by providing execution guarantees while maintaining the overall efficiency of SMT, embodiments of the present invention bridge the gap between high-throughput and time-critical computing paradigms.

Claims

Claims:

1. A processor capable of scheduling operations from multiple threads onto multiple functional units in the same cycle comprising: a policy engine that selects one or more active threads to be in a time-critical mode; a scheduler that selects operations for execution in each cycle from a subset of a plurality of active threads based on input from the policy engine; wherein the operation of the scheduler guarantees that the at least one time-critical thread meets a performance target.

2. The processor of claim 1 further comprising: a slot management module coupled to the scheduler wherein the scheduler is configured to allocate to each of the one or more time-critical threads fixed numbers of execution slots.

3. The processor of claim 2 wherein the processor implements a deterministic execution mode for at least one of the one or more time-critical threads.

4. The processor of claim 1 further comprising: a thread progress monitor module coupled to the scheduler wherein the scheduler dynamically adjusts the priority of at least one of the one or more time-critical threads based on a measured progress.

5. The processor of claim 4 wherein the processor implements a guaranteed worst case execution time (WCET) for at least one of the one or more time-critical threads.

6. The processor of claim 1 wherein the policy engine selects one or more active threads to be in a best-effort mode.

7. The processor of claim 1 further comprising: a resource management unit coupled to the scheduler, wherein the resource management unit partitions shared resources among the plurality of active threads to prevent interference with the at least one time-critical thread.

8. The processor of claim 7 wherein the shared resources include one or more of: I / O devices, buses, interconnects, and portions of memory hierarchies.

9. The processor of claim 7 wherein the resource management unit further comprises a monitoring and accounting engine that tracks resource usage patterns for the at least one time-critical thread.

10. The processor of claim 7 wherein the resource management unit further comprises a virtualization interface enabling dynamic resource allocation between time-critical and best-effort threads while maintaining timing guarantees.

11. A resource management unit comprising: a resource mapping unit that maps logical resource identifiers from each task or virtual machine to shared physical resources; an access control engine that enforces isolation policies and access rights; and a QoS policy module that maintains performance guarantees by regulating access bandwidth or latency.

12. The processor of claim 7 wherein the shared physical resources include one or more of: I / O devices, buses, interconnects, and portions of memory hierarchies.

13. The resource management unit of claim 11 further comprising: a monitoring and accounting engine that tracks usage patterns of shared resources.

14. The resource management unit of claim 11 further comprising: an interrupt and exception router that routes and virtualizes interrupts per execution context to maintain isolation between time-critical and non-time-critical threads.

15. The resource management unit of claim 10 wherein execution of time-critical threads is supported by guaranteeing compliance with timing constraints while allowing for controlled resource sharing.

Images