Dynamic Co-scheduling of Hardware Contexts for Parallel Runtime Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current operating systems and runtime systems perform poorly with parallel workloads that have variable CPU demands and frequent synchronization between threads, leading to issues like preemption at inconvenient times, load imbalances, and inefficient resource utilization on multi-core machines.

Innovation Solution

Implementing a dynamic co-scheduling system that manages hardware contexts through a resource management layer between the operating system and parallel runtime systems, allowing for dynamic allocation and re-allocation of hardware threads based on application demands, using spatial and temporal scheduling policies to reduce load imbalances and optimize resource use.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple parallel applications are run on shared hardware contexts, then resource utilization improves, but load imbalances and interference between applications increase

Engineering Contradiction:

Improveresource utilizationVSAvoidload balance

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements dynamic co-scheduling that allows hardware contexts to be dynamically allocated and re-allocated between parallel applications based on their current resource demands. The system monitors application workload and adjusts context assignment in real-time, transitioning from static to dynamic resource management to maintain load balance while maximizing utilization.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the scheduling parameters by introducing spatial scheduling (which hardware context runs which application) and temporal scheduling (when to re-allocate contexts) policies. These parameter changes enable the system to optimize both resource utilization and load balance by adjusting context assignment based on application phase and workload characteristics.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If hardware contexts are dynamically re-allocated between applications, then resource efficiency improves, but preemption at inconvenient times increases

Engineering Contradiction:

Improveresource efficiencyVSAvoidpreemption timing

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements check-in operations that allow applications to proactively signal when they are ready to yield hardware contexts. By performing preliminary actions (check-ins) at convenient times, applications can coordinate context re-allocation before it occurs, avoiding preemption during critical sections and ensuring smooth transitions.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback mechanisms where applications report their readiness state and resource demands to the scheduler. This feedback loop enables the scheduler to make informed decisions about when to re-allocate contexts, choosing times that are convenient for applications rather than imposing arbitrary preemption schedules.

Inventive Principle:

Principle #23Feedback

3Device complexity

If traditional operating systems manage parallel workloads, then system simplicity is maintained, but performance with variable CPU demands deteriorates

Engineering Contradiction:

Improvesystem structureVSAvoidworkload performance

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The patent introduces a resource management layer as an intermediary between the operating system and parallel runtime systems. This intermediary component handles the complex tasks of dynamic context allocation and coordination, allowing the OS to remain simple while enabling sophisticated resource management for parallel workloads with variable CPU demands.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10241831B2Dynamic co-scheduling of hardware contexts for parallel runtime systems on shared machines

Publication Date: 2019.03.26 ORACLE INT CORP

AI summary

Multi-core computers may implement a resource management layer between the operating system and resource-management-enabled parallel runtime systems. The resource management components and runtime systems may collectively implement dynamic co-scheduling of hardware contexts when executing multiple parallel applications, using a spatial scheduling policy that grants high priority to one application per hardware context and a temporal scheduling policy for re-allocating unused hardware contexts. The runtime systems may receive resources on a varying number of hardware contexts as demands of the applications change over time, and the resource management components may co-ordinate to leave one runnable software thread for each hardware context. Periodic check-in operations may be used to determine (at times convenient to the applications) when hardware contexts should be re-allocated. Over-subscription of worker threads may reduce load imbalances between applications. A co-ordination table may store per-hardware-context information about resource demands and allocations.