Concurrency Reduction Service for Microservices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microservices architecture, multiple instances of a service concurrently requesting the same data from a back-end service lead to redundant requests, causing unnecessary stress and delays, which existing caching and proxy technologies do not adequately address, especially for initial requests.
Innovation Solution
Implementing a concurrency reduction service that maps incoming requests to keys, groups matching requests, and sends a single request to the back-end service, ensuring only one response is received and reused for all matching requests, thereby reducing redundant traffic before the first response is received.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple service instances concurrently request the same data from a back-end service, then each service instance can obtain the required data, but redundant requests cause unnecessary stress and delays on the back-end service
Solution Approach 1:
The patent introduces a concurrency reduction service as an intermediary component between service instances and the back-end service. This mediator intercepts concurrent requests, identifies duplicates through key mapping, and consolidates them into single requests before forwarding to the back-end service, thereby reducing stress while ensuring data retrieval reliability
Solution Approach 2:
The patent merges multiple identical concurrent requests into a single request by mapping them to the same key. When the concurrency reduction service detects that multiple requests share the same key, it consolidates them and sends only one request to the back-end service, eliminating redundant traffic and improving back-end performance
2Speed
If caching is used to store retrieved data, then subsequent requests can be served faster, but caching does not prevent redundant requests from reaching the back-end service for initial data retrieval
Solution Approach 1:
The patent performs preliminary action by consolidating concurrent requests before they reach the back-end service. The concurrency reduction service proactively identifies and merges duplicate requests using key mapping, preventing redundant requests from being generated in the first place, rather than merely caching responses after retrieval
Data Source
AI summary
Reducing concurrency in a system comprising a plurality of services sending a plurality of concurrent requests to a back-end service through a concurrency reduction service. The concurrency reduction service maps each concurrent incoming request to a key; and determines if the key to which the concurrent incoming request is mapped matches a key mapped from another concurrent incoming request from the plurality of concurrent requests. If the key to which the concurrent incoming request is mapped matches a key mapped from at least one other concurrent incoming request from the plurality of concurrent requests, then the service groups the concurrent incoming request with the at least one other concurrent incoming request to be sent to the back-end service. The concurrency reduction service sends one outgoing request to the back-end service for each of the keys to which the plurality of concurrent incoming requests were mapped.