Polling Resource Manager for Web Service Latency

Solution Overview

Problem

Existing web service systems face inefficiencies in managing polling resources, leading to increased latency due to static allocation, which fails to handle varying message traffic effectively, resulting in under-servicing or over-servicing of message queues.

Innovation Solution

Implementing a polling resource manager that automatically scales polling resources such as polling assignments, threads, and invoke threads based on message traffic patterns, utilizing autoscaling routines to dynamically adjust resource quantities and distribution across hosts, optimizing throughput and resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If static allocation of polling resources is used, then resource allocation is simple and stable, but latency increases and resource utilization is inefficient under varying traffic conditions

Engineering Contradiction:

ImprovelatencyVSAvoidpolling resource management complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent implements dynamic polling resource allocation by continuously monitoring message queue depths and adjusting the number of polling threads and assignments in real-time. This allows the system to adapt to varying traffic conditions, reducing latency during high traffic periods while maintaining efficiency during low traffic periods, thereby resolving the contradiction between static simplicity and dynamic performance.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system employs feedback mechanisms by monitoring message queue depths, polling success rates, and traffic patterns, then using this information to automatically adjust polling resource allocation. This closed-loop control enables the system to respond to changing conditions and optimize latency performance without manual intervention, balancing the trade-off between complexity and time loss.

Inventive Principle:

Principle #23Feedback

2Productivity

If static allocation of polling resources is used, then system operation is simple, but resource utilization is inefficient leading to under-servicing or over-servicing of message queues

Engineering Contradiction:

Improveresource utilization efficiencyVSAvoidpolling resource management complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent implements dynamic polling resource allocation by continuously monitoring message queue depths and adjusting the number of polling threads and assignments in real-time. This allows the system to adapt to varying traffic conditions, reducing latency during high traffic periods while maintaining efficiency during low traffic periods, thereby resolving the contradiction between static simplicity and dynamic performance.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes key parameters such as the number of polling threads, polling assignments, and scaling factors based on monitored performance metrics. By dynamically adjusting these parameters according to traffic conditions, the system optimizes resource utilization efficiency, preventing both under-servicing and over-servicing of message queues.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If polling resources are increased to handle high traffic, then message processing capability improves, but resource waste occurs during low traffic periods

Engineering Contradiction:

Improvemessage processing capabilityVSAvoidresource waste

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent implements dynamic polling resource allocation by continuously monitoring message queue depths and adjusting the number of polling threads and assignments in real-time. This allows the system to adapt to varying traffic conditions, reducing latency during high traffic periods while maintaining efficiency during low traffic periods, thereby resolving the contradiction between static simplicity and dynamic performance.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system dynamically scales polling resources up during high traffic periods and scales them down during low traffic periods. By recovering and re allocating polling threads and assignments based on current demand, the system avoids resource waste during low traffic while maintaining high processing capability when needed, effectively balancing productivity and energy loss.

Inventive Principle:

Principle #34Discarding and recovering

Data Source

PatentUS10979359B1Polling resource management system

Publication Date: 2021.04.13 AMAZON TECH INC

AI summary

A polling resource management system and methodology for facilitating the management of polling resources for polling message queues. A user interacts with customer applications within a web services system. A polling manager within the web services system utilizes polling resources, such as polling threads, polling assignments, and polling workers to access and implement messages in the message queues. A polling resource manager automatically scales the polling assignments, polling threads, and invoke threads to better utilize the polling resources and to reduce latency experienced by the user when interacting with the customer applications.