Service Orchestrator Estimated Response Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microservice-based systems, the execution of service sequences can be hindered by high load metrics on computing devices and communication networks, leading to delays or failures in obtaining real responses, which affects performance and responsiveness.
Innovation Solution
A service orchestrator selects services in a sequence and detects load metrics; if the load exceeds a threshold, it identifies deviation behavior identifiers to obtain estimated responses using static rules or machine learning models, allowing the execution to continue with pre-computed or generated responses, thereby bypassing delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the service orchestrator executes services in sequence to obtain real responses, then the accuracy and reliability of service execution is improved, but the system performance and responsiveness deteriorate when load metrics exceed thresholds due to delays and failures
Solution Approach 1:
The system pre-computes and stores estimated responses for services before they are needed during high-load periods. When a service is selected by the orchestrator, if the load metric exceeds the threshold, the pre-computed estimated response is immediately retrieved and used, eliminating the need to wait for real-time service execution and thus maintaining system responsiveness while ensuring service sequence continuity
Solution Approach 2:
An estimated response mechanism is introduced as an intermediary between the service orchestrator and the actual service execution. This intermediary provides substitute responses when real service execution is delayed or failed due to high load, allowing the service sequence to continue without interruption and maintaining overall system productivity and responsiveness
2Measurement precision
If the service orchestrator waits for real responses from services, then the accuracy of service execution is improved, but the execution time and service sequence completion time increase due to resource-intensive computations and network delays
Solution Approach 1:
Estimated responses are pre-computed and stored in advance based on historical data, service patterns, and input parameters. When the orchestrator needs a service response and the load metric exceeds the threshold, the pre-computed estimated response is immediately retrieved and used, eliminating the time-consuming wait for real service execution while providing sufficiently accurate results for service sequence continuity
Solution Approach 2:
The system changes the parameter of response type from 'real response' to 'estimated response' based on load conditions. When load metrics exceed thresholds, the orchestrator switches to using estimated responses which are pre-computed with sufficient accuracy for service sequence decision-making, thus reducing execution time while maintaining acceptable precision for operational continuity
3Productivity
If the service orchestrator executes all services in the predefined sequence, then the completeness of service execution is improved, but the system load and resource consumption increase, leading to delays and failures
Solution Approach 1:
The estimated response mechanism acts as an intermediary that reduces system load by avoiding actual service execution when load metrics exceed thresholds. The orchestrator uses these intermediary estimated responses to maintain service sequence completeness and productivity while preventing system overload, thus resolving the contradiction between execution completeness and system stress
Solution Approach 2:
The system applies partial action by using estimated responses instead of full service execution when load conditions warrant it. This partial substitution maintains the appearance of complete service sequence execution while actually reducing resource consumption and system load, preventing delays and failures while preserving service execution completeness from the orchestrator's perspective
Data Source
AI summary
A method for executing a service sequence, comprising: selecting, by a service orchestrator, a service that is part of the service sequence; detecting, by the service orchestrator, whether a load metric exceeds a threshold; when the load metric does not exceed the threshold, executing the service to obtain a real response, and continuing execution of the service sequence based on the real response; when the load metric exceeds the threshold, identifying, by the service orchestrator, a deviation behavior identifier that is associated with the service, obtaining an estimated response for the service based on the deviation behavior identifier, and continuing execution of the service sequence based on the estimated response.