Chaos Engineering in Microservices via Service Mesh
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microservices-based SaaS systems, traditional approaches to increasing availability focus on extending mean time to failure, which can lead to risk aversion and hinder system changes, while mean time to diagnose and repair remains high due to complexity and manual processes, necessitating a more efficient method to reduce downtime and improve resilience.
Innovation Solution
Implementing chaos testing with a service mesh to prioritize and automate failure injection based on API dependency analysis, using probabilistic chaos components to generate graphs and assign weights to microservices, allowing for targeted and efficient failure testing to reduce mean time to repair and improve system resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional approaches focus on extending mean time to failure to increase availability, then system reliability is improved, but risk aversion increases and system changes are hindered
Solution Approach 1:
The patent applies preliminary action by performing chaos testing and failure injection before deploying system changes to production. The system automatically identifies critical microservices through API dependency analysis and pre-tests failure scenarios, allowing teams to validate resilience before making changes, thus enabling adaptability without compromising availability.
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring system behavior during chaos testing and using this information to improve system resilience. The automatic failure injection and monitoring provide real-time feedback on system weaknesses, enabling continuous improvement of reliability while maintaining adaptability through data-driven decisions.
2Productivity
If microservices system complexity increases to enable independent development and deployment, then productivity is improved, but mean time to diagnose and repair increases
Solution Approach 1:
The patent introduces an intermediary layer (service mesh with automated chaos testing) that mediates between microservices and failure conditions. This intermediary automatically performs failure injection and monitoring, reducing the time to diagnose issues by providing centralized visibility into service dependencies and failure propagation across the distributed system.
Solution Approach 2:
The patent replaces manual diagnosis and repair processes with automated chaos testing and failure injection mechanisms. The system automatically identifies failed microservices, traces dependency chains through API graphs, and triggers remediation actions, substituting manual mechanical processes with automated computational systems that reduce diagnostic time.
3Reliability
If chaos testing is applied to all microservices to improve system resilience, then reliability is improved, but testing time and resources increase significantly
Solution Approach 1:
The patent applies local quality by performing chaos testing selectively on critical microservices identified through API dependency analysis rather than uniformly across all services. The system calculates centrality scores and identifies high-impact services that require resilience testing, concentrating testing resources where they provide maximum value to system reliability while minimizing overall testing time.
Solution Approach 2:
The patent implements partial action by performing chaos testing on a prioritized subset of microservices based on their criticality to system functionality. Rather than testing all services equally, the system applies excessive testing focus to high-impact services while using automated dependency analysis to infer resilience characteristics of less critical services, reducing total testing time while maintaining adequate system-wide resilience.
Data Source
AI summary
Techniques are provided for chaos engineering in microservices using a service mesh. In an example, a plurality of microservices can operate together as part of a software as a service product. A graph of the service mesh of the plurality of microservices can be determined. From that graph, weight can be assigned to the respective nodes. Those weights can be used to determine a probability of where chaos is introduced in the corresponding microservice architecture as part of chaos testing.