Local Queue Fiber Management for Asynchronous Call Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems managing asynchronous functions in computing devices face inefficiencies due to excessive use of L2 cache and memory, leading to slow responsiveness and high power consumption, as fibers waiting for results are often queued in a global queue and may be pushed out of cache before being resumed.
Innovation Solution
Implementing a method where an operating system pauses fibers waiting for asynchronous function results, enqueues them in a local queue, and resumes execution when results are completed, with the option to prioritize execution from a local queue over a global queue, reducing reliance on L2 cache and memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fibers are enqueued in a global queue to wait for asynchronous function results, then system responsiveness is maintained through centralized management, but cache locality deteriorates and power consumption increases due to frequent L2 cache and memory access
Solution Approach 1:
The patent segments the global queue into multiple per-core local queues, assigning each fiber to a local queue based on its waiting asynchronous function. This segmentation keeps frequently accessed queue data in L1 cache per core, eliminating the need for cores to access L2 cache or memory for queue operations, thereby reducing power consumption while maintaining responsive fiber resumption.
Solution Approach 2:
The patent implements local quality by creating per-core local queues with different characteristics from the global queue. Each local queue is optimized for its specific core's cache hierarchy, ensuring that queue data remains in L1 cache for fast access. This local optimization reduces the power-consuming L2 cache and memory accesses that occur with a centralized global queue.
2Device complexity
If fibers are enqueued in a global queue to wait for asynchronous function results, then centralized queue management is achieved, but processing speed decreases due to slow memory access
Solution Approach 1:
The patent divides the centralized global queue into distributed per-core local queues. This segmentation allows each core to manage its own queue in L1 cache, eliminating the bottleneck of global queue access that requires L2 cache or memory operations. The result is significantly faster fiber resumption and asynchronous function completion handling.
Solution Approach 2:
The patent transitions from a single-dimensional global queue structure to a multi-dimensional structure with per-core local queues. This dimensional change organizes queue data across the cache hierarchy in a way that exploits spatial and temporal locality, keeping related data in L1 cache and improving processing speed without increasing overall system complexity.
3Device complexity
If fibers are enqueued in a global queue to wait for asynchronous function results, then unified queue management is maintained, but cache locality is reduced leading to excessive L2 cache usage
Solution Approach 1:
The patent segments the global queue into per-core local queues, distributing queue data across multiple L1 caches instead of concentrating it in a single global queue that must be accessed through L2 cache. This segmentation dramatically reduces L2 cache usage by keeping queue data local to each core's L1 cache, where it can be accessed without L2 cache involvement.
Solution Approach 2:
The patent applies local quality by creating per-core local queues that are optimized for their specific core's cache hierarchy. Each local queue resides in the core's L1 cache, ensuring maximum cache locality and minimizing L2 cache usage. This local optimization maintains effective queue management while dramatically reducing the volume of data that must be stored in L2 cache.
Data Source
AI summary
This disclosure generally relates to operating systems and methods of computing devices for managing system and function calls. An example method include determining that a fiber is requesting to wait for one or more results of an asynchronous function call, pausing execution of the fiber until the one or more results are completed, enqueuing the paused fiber in a local queue of the one or more results, determining that the one or more results are completed, accessing one or more queued fibers in the local queue of the one or more results. The one or more queued fibers comprise the fiber and resuming execution of the one or more queued fibers, and the asynchronous function call is called by a thread to execute a task without being blocked while the task is being completed.