VM Load Balancer for Deterministic Signaling Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In virtual machine environments like Java, C#, and Lisp, 'stop the world' garbage collection events cause application processing delays, making it challenging to provide deterministic latency for communication signaling protocols like SIP, especially when the object creation rate exceeds the capacity of periodic garbage collection threads.

Innovation Solution

A load balancer monitors heap consumption and garbage collection activity, throttling request acceptance to maintain deterministic latency by regulating the rate of request processing based on specified latency thresholds, using functions to predict garbage collection latency and object reclamation rates, and prioritizing message handling to optimize garbage collection activity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If periodic garbage collection thread is used to manage memory, then automatic memory management is maintained, but deterministic latency cannot be guaranteed when object creation rate exceeds collection capacity

Engineering Contradiction:

Improvedeterministic latencyVSAvoidobject creation rate

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system dynamically adjusts the garbage collection frequency and aggressiveness based on real-time workload conditions. When object creation rate exceeds collection capacity, the system increases collection frequency or uses more aggressive collection strategies to maintain deterministic latency guarantees.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes garbage collection parameters such as frequency, threshold values, and collection algorithms based on monitored object creation rates and heap consumption patterns, allowing adaptation to varying workload conditions while maintaining latency guarantees.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If CPU capacity is increased to handle garbage collection, then deterministic latency is improved, but system cost increases

Engineering Contradiction:

Improvedeterministic latencyVSAvoidCPU capacity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system applies partial garbage collection actions by collecting only when object creation rate exceeds collection capacity, rather than continuously collecting. This reduces the total CPU capacity required while still maintaining deterministic latency guarantees when needed.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system monitors its own garbage collection performance and automatically adjusts collection frequency and parameters without external intervention, optimizing CPU usage while maintaining latency guarantees.

Inventive Principle:

Principle #25Self-service

3Reliability

If throughput is reduced to maintain deterministic latency, then latency reliability is improved, but system performance decreases

Engineering Contradiction:

Improvedeterministic latencyVSAvoidthroughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system dynamically adjusts throughput by modulating the acceptance of new objects or requests based on current garbage collection capacity and latency requirements, allowing maximum throughput when conditions permit while guaranteeing deterministic latency when needed.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS7730269B2Load management to reduce communication signaling latency in a virtual machine environment

Publication Date: 2010.06.01 X CORP

AI summary

A computer implemented method for reducing communication signaling protocol latency. An acceptable level of latency is specified. Automatic memory management activities are monitored based on specified parameters to calculate a level of activity that determines whether a reduction of activity is required.