Self-Healing Stateless Microservice Nodes via Randomized Reset Timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network management systems are vulnerable to failures due to memory leaks, file handle leaks, and other issues that can lead to service degradation or complete failure, and predictive methodologies are costly, time-consuming, and prone to false positives, especially during usage spikes.
Innovation Solution
A method where individual processes in a service cluster randomly generate and broadcast reset times, allowing them to self-heal by shutting down and restarting independently without disrupting the overall cluster, using a one-way datagram communication algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If individual processes are reset frequently to prevent memory leaks and failures, then service reliability is improved, but service downtime increases
Solution Approach 1:
The system implements periodic reset actions for microservice nodes, where each node is randomly selected and reset at scheduled intervals. This periodic action prevents memory leaks and service degradation while maintaining overall service availability through the stateless nature of the nodes and load balancer redistribution.
Solution Approach 2:
The system changes the operational state parameter of selected microservice nodes from active to reset state temporarily. By randomly selecting nodes and resetting them individually rather than all nodes simultaneously, the system maintains service continuity while preventing failures through periodic parameter changes.
2Reliability
If all microservice nodes are reset simultaneously to ensure complete service renewal, then service reliability is improved, but service availability deteriorates
Solution Approach 1:
The system segments the reset operation into individual node-level actions rather than a cluster-wide simultaneous operation. Each microservice node is independently selected and reset, allowing other nodes to continue serving requests. This segmentation maintains service availability while achieving reliability through individual node renewal.
Solution Approach 2:
The system implements dynamic node selection for resets using random selection rather than static scheduled resets for all nodes. This dynamic approach allows the system to adaptively choose which nodes to reset based on current state, preventing simultaneous resets that would cause service unavailability while maintaining reliability through continuous node renewal.
3Measurement precision
If predictive methodologies are used to determine reset timing, then reset precision is improved, but system complexity and cost increase
Solution Approach 1:
The system implements self-service through random node selection and automatic reset execution without requiring complex predictive analytics or manual intervention. Each node independently receives reset commands based on random selection, and the load balancer automatically redistributes traffic, eliminating the need for complex prediction systems while maintaining effective service renewal.
Data Source
AI summary
A computer-implemented method includes: randomly generating, by a process running on a computer device, a reset time for the process; broadcasting a message to other processes in a cluster, the message including the reset time; entering a running state; receiving another message from one of the processes, the other message including another reset time; comparing the other reset time to the reset time; and in response to the comparing, the process performs one of: continuing in the running state based on the other reset time being different than the reset time; and randomly generating a new reset time for the process based on the other reset time coinciding with the reset time.