Microservice Latency Tracking via Distributed In-Memory Key-Value Database
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Microservice-based architectures experience increased latency and management challenges due to the growing number of interactions and communication between services, which can lead to slower processing times and reduced productivity, necessitating improved methods for monitoring and measuring latency to optimize system performance.
Innovation Solution
A system and method utilizing a distributed in-memory key-value database with a publisher-subscriber function to track and measure latency across microservices, employing a switch API and configuration tools to collect and analyze data, and feeding it into an ASIC, while allowing for flexible scaling and deployment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of microservices is increased to provide more functionality, then the system's adaptability and versatility are improved, but the latency and processing time increase
Solution Approach 1:
The patent segments the monitoring system into distributed agents deployed across individual microservices, each independently collecting and processing latency data. This segmentation allows the system to handle increased microservice counts without centralizing bottlenecks, thereby maintaining low latency while supporting enhanced system functionality through additional services.
Solution Approach 2:
The patent introduces a new dimensional approach by implementing multi-dimensional latency tracking that captures timing data across different axes (service-to-service, request-type, time-period). This dimensional expansion enables the system to manage complex microservice interactions efficiently, providing comprehensive latency visibility without increasing overall system latency.
2Adaptability or versatility
If the number of microservices is increased to enhance system capabilities, then the adaptability is improved, but the device complexity and management difficulty increase
Solution Approach 1:
The patent implements a universal monitoring framework that can track latency across diverse microservice types and communication patterns through a single standardized system. The multi-functional agents can collect, process, and report latency data for various service interactions, reducing management complexity despite increased system capabilities and microservice diversity.
Solution Approach 2:
The system incorporates continuous feedback loops where latency measurements are collected, analyzed, and used to dynamically adjust monitoring parameters and alert thresholds. This feedback mechanism automates management tasks, reducing the complexity of managing expanded microservice architectures by providing self-adjusting monitoring behavior.
3Measurement precision
If comprehensive latency monitoring is implemented across all microservices, then the measurement precision is improved, but the device complexity and resource requirements increase
Solution Approach 1:
The monitoring system employs self-service agents deployed within each microservice that autonomously collect latency data without requiring external intervention. These agents automatically instrument services, collect timing information, and report to the central system, achieving high measurement precision while minimizing the complexity of the external monitoring infrastructure.
Solution Approach 2:
The patent introduces intermediary agents that act as mediators between microservices and the central monitoring system. These agents buffer, aggregate, and pre-process latency data before transmission, reducing the complexity of direct comprehensive monitoring while maintaining measurement precision through systematic data collection and reporting.
Data Source
AI summary
A system and method for measuring latency is disclosed. In some implementations, the processor may include providing a switch API to process a plurality of microservices involved in a workflow and connected to a distributed in-memory keyvalue database, where the distributed in-memory keyvalue database further may include a plurality of key spaces. In addition, the processor may include managing configuration tools to track the latency of each microservice in the plurality of microservices. The processor may include programing data plane information of the microservices to create programmed data, where programming further may include collecting and analyzing data on the latency of each microservice of the plurality of microservices. Moreover, the processor may include feeding the programmed data into an ASIC.