Dynamic Steering for HTTP Calls in Kubernetes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microservices-based Kubernetes cluster environments, implementing a controlled canary deployment of new application versions is challenging due to the need for manual control of replica ratios and limitations in traffic management tools like Istio.
Innovation Solution
The solution involves selectively steering data traffic in a Kubernetes cluster environment to multiple versions of a microservice based on information in the data message, using a proprietary HTTP header to specify the target microservice version and traffic steering mode.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual control of replica ratios is used to limit traffic to canary release, then traffic management is achieved, but operational complexity increases
Solution Approach 1:
The patent introduces a service mesh as an intermediary layer between clients and microservices. This service mesh automatically manages traffic routing based on headers, versions, and routing rules, eliminating the need for manual replica ratio control while maintaining reliable traffic management. The service mesh acts as a mediator that handles the complexity of canary deployments, A/B testing, and version routing automatically.
2Ease of operation
If Istio service mesh is used for traffic management, then separation between replica counts and traffic management is achieved, but traffic steering granularity is limited
Solution Approach 1:
The patent implements dynamic traffic steering by evaluating HTTP request headers in real-time and dynamically routing requests to appropriate microservice versions. The routing decisions are made at request level rather than relying on static replica ratios, enabling fine-grained control where each request can be independently directed based on its headers, user identity, or other dynamic criteria.
Solution Approach 2:
The patent enables different routing behaviors for different requests by examining local characteristics of each HTTP request (headers, cookies, user agents). This allows specific users or user groups to be directed to specific microservice versions based on their individual request properties, providing localized and granular traffic steering control rather than uniform routing.
3Reliability
If traditional canary deployment is used, then controlled percentage traffic shift is achieved, but fine-tuned steering based on user identity is not possible
Solution Approach 1:
The patent implements preliminary routing rule configuration where routing decisions are predetermined based on header patterns, user identities, or request characteristics. Before actual traffic flows, routing rules are established that specify which users or request types should be directed to which microservice versions. This preliminary configuration enables both controlled deployment and user-specific steering simultaneously.
Data Source
AI summary
Techniques are disclosed for deploying, by a packet processing system in a cloud computing environment, microservice instances in the cloud computing environment comprising a plurality of computing devices executing a plurality of Kubernetes clusters comprising one or more containers. The packet processing system is configured to selectively steer data traffic between multiple versions of microservice instances executing in the cloud computing environment.