Shard-Based Traffic Routing for Microservice API Facets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional microservice architectures lack fine-grained control over load balancing and auto-scaling, failing to meet distinct service level objectives for different API facets, leading to inefficient resource allocation and potential service degradation under varying traffic conditions.
Innovation Solution
Associating metadata with application interface specifications to configure shards for managing traffic, allowing for fine-grained service level objective management, including annotations for load balancing and auto-scaling, which enables routing traffic to dedicated replicas based on defined service level objectives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional load balancing and auto-scaling are used in microservice architectures, then traffic is distributed across replicas, but fine-grained service level objectives for different API facets cannot be met
Solution Approach 1:
The patent segments traffic management by introducing shards that divide traffic into separate streams based on API facets. Each shard can have its own load balancing and auto-scaling policies, enabling fine-grained service level objective management for different API paths, methods, or parameters while maintaining overall system coherence.
Solution Approach 2:
The patent introduces an intermediary component (shard manager or routing layer) that sits between the traditional load balancer and microservice instances. This intermediary captures traffic, applies metadata-based routing rules, and directs traffic to appropriate shards or replicas, enabling fine-grained control without fundamentally redesigning the entire architecture.
2Reliability
If replicas are continuously running to meet service level objectives, then service availability is maintained, but resource allocation becomes inefficient and costs increase
Solution Approach 1:
The patent implements dynamic replica management where the number and configuration of replicas are adjusted in real-time based on actual traffic conditions and service level objective requirements. Metadata annotations guide the system to scale replicas only when and where needed, rather than maintaining static replica counts across all services.
Solution Approach 2:
The patent changes the parameters of replica deployment by using metadata annotations that specify service level objectives for different API facets. The system monitors these parameters and dynamically adjusts replica configuration (number, size, placement) to meet the specified objectives while optimizing resource utilization and reducing unnecessary resource consumption.
3Reliability
If traffic is routed to dedicated replicas, then service level objectives are met, but replica startup time increases and cost optimization is reduced
Solution Approach 1:
The patent applies preliminary action by pre-configuring shards with metadata annotations that define service level objectives and routing rules. When traffic patterns indicate the need for dedicated replicas, the system can rapidly provision them because the shard configuration and routing logic are already in place, reducing startup time compared to creating entirely new service instances.
Solution Approach 2:
The patent makes replicas multi-functional by allowing them to serve multiple API facets through the shard routing mechanism. A single replica can handle traffic for different services or API paths by receiving routed traffic from the shard layer, reducing the total number of replicas needed and enabling faster deployment since replicas aren't dedicated to single functions.
Data Source
AI summary
Concepts for controlling interaction with scalable applications are presented. One example comprises associating metadata with an application interface specification of an application. Based on the metadata associated with the application interface specification of the application, configuration of shards for managing traffic of the interaction and/or integration with the application interface is controlled. Traffic is then routed to configured shards.