Microservice Routing With Nanosecond-Level Time Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time inaccuracies in data centers lead to inefficiencies and increased latencies in microservice architectures, causing a snowball effect that worsens overall performance by delaying dependent microservices.
Innovation Solution
Implementing precise and accurate time management systems for microservice routing, processing, and load balancing using hybrid hypervisors like SmartNIC/IPU/DPU, which enable nanosecond-level scheduling and cache optimization, improving deterministic processing and reducing tail latencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional time management systems are used in data centers, then system complexity is reduced, but time accuracy deteriorates leading to increased microservice latencies
Solution Approach 1:
The patent introduces a specialized time management intermediary layer that sits between the hybrid hypervisor and microservices. This intermediary handles precise time synchronization and timestamp generation, isolating the complexity of high-precision time management from the core microservice architecture while providing accurate time services to all components.
Solution Approach 2:
The patent replaces traditional software-based time management mechanisms with hardware-assisted timekeeping using TSC (Time Stamp Counter) and other hardware timers. This substitution provides nanosecond-level precision without the overhead and inaccuracies of software polling and synchronization mechanisms.
2Loss of time
If precise time management is implemented for microservice routing and scheduling, then microservice latency is reduced, but device complexity increases
Solution Approach 1:
The patent implements preliminary time window allocation where time slots are pre-calculated and assigned to microservices before execution. The hybrid hypervisor schedules microservices in predetermined time windows based on their priority and resource requirements, eliminating runtime scheduling decisions and reducing latency while keeping the scheduling logic manageable.
Solution Approach 2:
The patent changes the time measurement parameter from millisecond-level traditional timestamps to nanosecond-level precision using TSC counters. This parameter change enables fine-grained time-based scheduling and routing decisions without requiring a complete redesign of the time management infrastructure.
3Productivity
If time inaccuracies are not addressed, then system simplicity is maintained, but productivity deteriorates due to accumulated latencies and snowball effects
Solution Approach 1:
The patent implements feedback mechanisms where precise timestamps are recorded at each microservice processing stage. These timestamps are used to calculate actual processing times and latencies, which are then fed back to the hybrid hypervisor for dynamic scheduling adjustments. This feedback loop prevents latency accumulation by identifying and correcting timing issues in real-time.
Solution Approach 2:
The patent segments the microservice processing timeline into discrete, measurable time windows and intervals. Each microservice execution is bounded by precise start and end timestamps, allowing individual latency measurement and optimization. This segmentation prevents the snowball effect by isolating latency sources and enabling targeted remediation.
Data Source
AI summary
Systems and techniques for microservice latency reduction are described herein. A request may be received for execution of a microservice. An execution time may be calculated for the microservice. The execution time may be an estimation of time to complete execution of the microservice. A service level objective (SLO) may be identified. A processing unit of a computing node may be identified for execution of the microservice based on the precise execution time and the SLO. The microservice may be transmitted to the processing unit for instantiation.