Microservice Load Balancer Accelerator Offloading CPU Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In data centers using microservices, CPU utilization for communication between microservices introduces latency and inefficient resource use, particularly due to uneven core utilization and inefficient processing of HTTP2/HTTP3 requests with multiple object types, which can lead to performance degradation.

Innovation Solution

Implementing a load balancer accelerator and a queue system to allocate packet data for processing by processor cores and manage remote procedure calls, providing quality of service and reducing latency by balancing traffic and processing communications efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If CPU resources are utilized for communication between microservices, then communication can be processed, but latency increases and resource efficiency decreases

Engineering Contradiction:

Improvecommunication speedVSAvoidcommunication latency

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent segments communication handling by introducing dedicated network processors (NPs) that offload communication tasks from general-purpose CPU cores. This segmentation allows communication packets to be processed in parallel by specialized hardware, reducing CPU utilization and communication latency simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces network processors as intermediary components between network interfaces and application microservices. These NPs act as mediators that handle packet processing, buffering, and forwarding without requiring CPU intervention, thereby reducing both latency and CPU resource consumption.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If CPU resources are utilized for communication between microservices, then communication processing is enabled, but resource utilization efficiency deteriorates

Engineering Contradiction:

Improvecommunication processing capabilityVSAvoidCPU resource efficiency

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent implements self-service communication processing where network processors autonomously handle packet routing, buffering, and forwarding without CPU intervention. The NPs independently manage communication tasks using their own instruction sets and control logic, eliminating the need for CPU resources while maintaining full communication processing capability.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the software-based communication processing mechanism (CPU executing communication protocols) with hardware-based network processors. This substitution eliminates the overhead of operating system context switches and software protocol stacks, improving resource efficiency while maintaining processing capability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Ease of operation

If load balancing is implemented among CPU cores, then traffic distribution is improved, but processing speed decreases due to context switching

Engineering Contradiction:

Improvetraffic distributionVSAvoidprocessing speed

Core Design Contradiction:

Ease of operationVSSpeed

Solution Approach 1:

The patent segments the load balancing function by implementing dedicated network processors that independently manage traffic distribution. Each NP maintains separate packet buffers and processing queues, eliminating the need for CPU context switching while achieving effective load balancing across multiple NPs.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements periodic load balancing where network processors independently manage traffic in discrete time intervals without requiring CPU intervention. This periodic operation eliminates context switching overhead while maintaining balanced traffic distribution across the system.

Inventive Principle:

Principle #19Periodic action

4Reliability

If queue systems are implemented for RPC communications, then quality of service is improved, but system complexity increases

Engineering Contradiction:

Improvequality of serviceVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent implements self-service queue management where network processors autonomously handle RPC communication queuing, prioritization, and forwarding. The NPs independently manage multiple communication queues with different quality of service levels without requiring complex external control systems, thereby improving reliability while limiting complexity growth.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20230100935A1Microservice deployments using accelerators

Publication Date: 2023.03.30 INTEL CORP

AI summary

Examples described herein relate to circuitry to perform load balancing; at least one memory; and at least one processor. In some examples, at least one processor is to execute instructions stored in the at least one memory that cause the at least one processor to: execute a communication proxy that is to allocate packet data to the circuitry to perform load balancing to allocate workloads among cores and allocate received and transmitted remote procedure calls to at least one queue in circuitry to queue one or more packets.