API Wrapper Batching for Concurrent Request Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for processing API calls face challenges in supporting high concurrency and low latency, as conventional methods like using locks increase response time, load balancing consumes significant resources, and caching requires complex and resource-intensive code maintenance.
Innovation Solution
An API wrapper is introduced to receive and queue multiple API calls during a given time interval, sending a single call to the underlying API and providing the response to all callers, while utilizing a cache to ensure faster response times and data availability, even when the underlying API is slow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a lock is used to ensure sequential execution of API calls, then data consistency is maintained, but average response time increases significantly
Solution Approach 1:
The API wrapper performs preliminary actions by receiving and queuing multiple API calls before executing them. It collects all calls during a given time interval and sends them to the underlying API as a batch, then distributes the single response to all callers. This preliminary collection and batching approach maintains data consistency while significantly reducing response time compared to sequential execution with locks.
2Loss of time
If load balancing is used to distribute calls to different nodes, then response time is reduced, but computing resource consumption increases significantly
Solution Approach 1:
The API wrapper merges multiple API calls into a single batched call to the underlying API. By collecting all calls during a time interval and sending them as one request, it eliminates the need for distributed load balancing across multiple nodes. This merging approach reduces response time while significantly lowering computing resource consumption compared to distributing calls across multiple nodes.
3Loss of time
If a cache is used to store API responses, then response speed is improved, but code complexity and maintenance difficulty increase
Solution Approach 1:
The API wrapper performs self-service by automatically managing the caching mechanism. It receives calls, batches them, sends to the underlying API, and automatically distributes responses to all callers without requiring external cache management code. The wrapper handles the entire caching process internally, improving response speed while avoiding the code complexity and maintenance difficulties of traditional external cache systems.
4Productivity
If multiple API calls are sent concurrently to the underlying API, then productivity is improved, but the underlying API may become overloaded and unavailable
Solution Approach 1:
The API wrapper performs preliminary action by collecting and queuing multiple calls before sending them to the underlying API. It receives all calls during a given time interval, batches them into a single request, and then sends the batched calls to the underlying API. This preliminary collection and batching approach maintains high productivity by processing multiple calls efficiently while preventing the underlying API from being overloaded, thus ensuring reliability.
Data Source
AI summary
The disclosure relates to processing application programming interface (API) requests. Embodiments include receiving, at an API wrapper, from a first caller, a first call to an API and sending the first call to the API. Embodiments include receiving, by the API wrapper, from one or more second callers, a second one or more calls to the API prior to receiving a response from the API to the first call. Embodiments include receiving, by the API wrapper, the response from the API to the first call and responding to the first call from the first caller with the response from the API to the first call. Embodiments include responding, by the API wrapper, to the second one or more calls from the one or more second callers with the response from the API to the first call without sending the second one or more calls to the API.