JIT System Converter Reordering Loads for Context Switching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current processor architectures face inefficiencies in context switching, particularly when handling a large number of threads, due to the need for duplicating architecture state elements and the overhead of thread synchronization, which hinders fine-grain parallelism and auto-parallelization of non-threaded software codes.

Innovation Solution

An agnostic runtime architecture system that employs a hardware accelerated conversion/JIT layer for efficient execution of guest code, utilizing system emulation and virtualization converters to reorder instructions and manage speculative architectural states, thereby reducing the need for context switching and enhancing parallel execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If hardware duplicates all architecture state elements for each supported thread, then context switching is eliminated for hardware-supported threads, but area, power consumption, and device complexity increase significantly

Engineering Contradiction:

Improvecontext switching efficiencyVSAvoidhardware state duplication

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent creates virtual copies of architecture state elements through software emulation rather than physical duplication in hardware. The system emulator maintains a virtual architecture state that can be copied and restored through software operations, eliminating the need for expensive hardware state duplication while achieving fast context switching for virtualized threads

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces hardware-based context management with software-based emulation. Instead of using hardware registers and state duplication mechanisms, the system uses a software system emulator that interprets and executes instructions while maintaining virtual state, substituting mechanical hardware operations with software-based state management

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If hardware supports multiple context states for a limited number of threads, then context switching is reduced, but the area and power consumption of duplicating architecture state elements increases

Engineering Contradiction:

Improvethread execution efficiencyVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system creates virtual copies of architecture state through software emulation, allowing multiple threads to share physical hardware resources while maintaining isolated virtual states. This approach eliminates the need for power-intensive hardware state duplication while enabling efficient thread context management through software-based state copying and restoration

Inventive Principle:

Principle #26Copying

3Productivity

If software threads are used for fine-grain parallelism, then parallel execution capability increases, but thread synchronization overhead and context switching cost increase

Engineering Contradiction:

Improveparallel execution capabilityVSAvoidsynchronization overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent introduces a system emulator as an intermediary layer between the hardware and software threads. The emulator manages thread contexts and synchronization operations, providing efficient mediation that reduces direct thread synchronization overhead while maintaining fine-grain parallelism through virtualized thread execution

Inventive Principle:

Principle #24Intermediary (Mediator)

4Reliability

If context switch includes saving and restoring architecture state, then thread isolation is maintained, but execution time increases due to hundreds of cycles required

Engineering Contradiction:

Improvethread state isolationVSAvoidcontext switch time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system uses efficient software-based copying of architecture state through the system emulator. Instead of costly hardware state saving and restoration, the emulator maintains virtual state representations that can be quickly copied and restored through software operations, reducing context switch time while preserving thread isolation

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs preliminary state preparation and caching within the emulator to enable faster context restoration. The system emulator pre-manages architecture state representations and prepares them for rapid switching, reducing the actual context switch time while maintaining complete thread state isolation

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9733909B2System converter that implements a reordering process through JIT (just in time) optimization that ensures loads do not dispatch ahead of other loads that are to the same address

Publication Date: 2017.08.15 INTEL CORP

AI summary

A system for an agnostic runtime architecture. The system includes a system emulation/virtualization converter, an application code converter, and a converter wherein a system emulation/virtualization converter and an application code converter implement a system emulation process, and wherein the system converter implements a system and application conversion process for executing code from a guest image, wherein the system converter or the system emulator. The system further includes a reordering process through JIT (just in time) optimization that ensures loads do not dispatch ahead of other loads that are to the same address, wherein a load will check for a same address of subsequent loads from a same thread, and a thread checking process that enable other thread store checks against the entire load queue and a monitor extension.