Virtual Hardware Thread Scheduling for Cloud VNF Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud scaling technologies face challenges in efficiently managing thousands of Virtual Network Functions (VNFs) due to high scheduling and CPU oversubscription overheads, leading to resource drain and latency issues, especially when using thin virtual machines for microservices and web-services.
Innovation Solution
The implementation of non-blocking, non-spinning cross-domain event synchronization and data communication using virtual hardware threads (VHTRs) that allow for efficient CPU resource allocation and scheduling, avoiding context switch penalties and operating system overheads by using a hardware scheduler to switch VHTRs in and out of execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If polling-mode interactions are used to implement event-driven state-engine traversals, then efficient event processing can be achieved, but significant CPU resources and power are drained due to continuous polling loops
Solution Approach 1:
The patent replaces the mechanical polling loop system with an event-driven architecture using hardware threads and interrupt-based notification. Instead of continuously polling for events, the system uses hardware threads that are automatically activated when events occur, eliminating the need for continuous CPU polling and reducing energy consumption while maintaining efficient event processing.
Solution Approach 2:
The patent implements periodic action through event-triggered activation of hardware threads. Rather than continuous polling, the system periodically activates hardware threads only when events occur, allowing the CPU to remain in low-power states between events while ensuring timely event processing when needed.
2Adaptability or versatility
If thin virtual machines for microservices are deployed to scale cloud services, then cloud scalability is improved, but scheduling overheads increase due to CPU pinning and context switching
Solution Approach 1:
The patent segments the CPU into multiple hardware threads that can be independently scheduled. This segmentation allows the system to manage thousands of VNFs by distributing them across multiple hardware threads, reducing the scheduling overhead that would otherwise occur with traditional CPU pinning and context switching mechanisms.
Solution Approach 2:
The patent introduces hardware threads as an intermediary between the CPU and VNFs. These hardware threads act as mediators that automatically manage context switching and scheduling, eliminating the need for complex software-based scheduling mechanisms and reducing overall scheduling overhead while enabling improved cloud scalability.
3Stability of the object's composition
If CPU pinning is used to schedule VNFs, then execution determinism is improved, but massive oversubscription occurs leading to resource exhaustion
Solution Approach 1:
The patent implements dynamic resource allocation through hardware threads that can be scheduled based on actual workload demands. Instead of static CPU pinning that causes oversubscription, the system dynamically allocates hardware threads to VNFs as needed, maintaining execution determinism while preventing resource exhaustion through flexible, demand-driven resource allocation.
Data Source
AI summary
Disclosed embodiments relate to cloud scaling with non-blocking, non-spinning cross-domain event synchronization and data communication. In an example, a processor includes a memory to store multiple virtual hardware thread (VHTR) descriptors, each including an architectural state, a monitored address range, a priority, and an execution state, fetch circuitry to fetch instructions associated with a plurality of the multiple VNFs, decode circuitry to decode the fetched instructions, scheduling circuitry to allocate and pin a VHTR to each of the plurality of VNFs, schedule execution of a VHTR on each of a plurality of cores, set the execution state of the scheduled VHTR; and in response to a monitor instruction received from a given VHTR, pause the given VHTR and switch in another VHTR to use the core previously used by the given VHTR, and, upon detecting a store to the monitored address range, trigger execution of the given VHTR.