Network Interface Device Selecting Services via TTL Load Balancing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud service providers face challenges in maintaining consistent latency for microservices deployed across different instances within a datacenter, as latency varies based on physical proximity, leading to inconsistencies that can breach service level agreements (SLAs).

Innovation Solution

A network interface device is configured to select a target service based on the number of permitted hops and Time To Live (TTL) using a load balancer, which can execute Kubernetes-based load balancing offloaded from the host CPU, ensuring that service instances comply with SLA tiers by choosing instances that minimize network hops.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If service instances are deployed across different physical locations in the datacenter, then service availability and load distribution are improved, but network latency becomes inconsistent and SLA compliance deteriorates

Engineering Contradiction:

Improveservice availabilityVSAvoidSLA compliance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system pre-establishes multiple service instances at different physical locations (same rack, different rack, different datacenter) before requests arrive. When a request comes in, the load balancer can immediately direct it to an appropriate pre-deployed instance based on TTL requirements, avoiding the need to dynamically provision or migrate services during runtime.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The load balancer acts as an intermediary between clients and service instances, intelligently routing requests based on TTL values. It mediates the trade-off between physical distribution (for availability) and proximity (for latency) by selecting appropriate instances that satisfy both requirements through TTL-based routing decisions.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Stability of the object's composition

If service instances are placed in different racks or datacenters, then system redundancy is improved, but network hop count increases and latency consistency worsens

Engineering Contradiction:

Improvesystem redundancyVSAvoidnetwork latency

Core Design Contradiction:

Stability of the object's compositionVSLoss of time

Solution Approach 1:

The system implements local quality by deploying service instances with different TTL characteristics at appropriate locations. Critical services requiring low latency are placed in the same rack (TTL=1), while less time-sensitive services can be distributed to different racks or datacenters. Each service instance is optimized for its specific location's latency characteristics.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The load balancer dynamically adjusts routing decisions based on real-time conditions and TTL requirements. It can shift traffic between locally-deployed instances (for low latency) and remotely-deployed instances (for load distribution and redundancy) depending on the specific request's TTL parameter and current system state.

Inventive Principle:

Principle #15Dynamics

3Productivity

If load balancing is implemented across distributed service instances, then service capacity is improved, but routing complexity and decision-making overhead increase

Engineering Contradiction:

Improveservice capacityVSAvoidrouting complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system uses TTL (Time To Live) as a key parameter to simplify routing decisions. Instead of complex multi-factor optimization, the load balancer primarily uses the TTL value embedded in requests to determine instance selection. This parameter-based approach scales efficiently while maintaining service capacity across distributed instances.

Inventive Principle:

Principle #35Parameter changes

4Speed

If service instances are co-located on the same server, then network latency is minimized, but resource utilization and load distribution deteriorate

Engineering Contradiction:

Improveservice response speedVSAvoidresource utilization

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

The system implements partial co-location by deploying critical service instances on the same server (achieving minimal latency) while also maintaining additional instances at other locations. The load balancer directs time-sensitive traffic to co-located instances (partial action for speed) while using remote instances for less urgent requests, achieving a balance between speed and resource utilization.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20230319133A1Network interface device to select a target service and boot an application

Publication Date: 2023.10.05 INTEL CORP

AI summary

Examples described herein relate to a network interface device that includes a network interface and circuitry. In some examples, the circuitry is to receive a request to perform a service and select a servicing node based on network latency and/or proximity of the requested service to the network interface device. In some examples, a proximity of the requested service includes execution in the network interface device.