Dual Stage Bulkheads for Web Service Request Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Single-threaded process architectures in web-based platforms face latency issues due to blocking on input/output operations, leading to delays in servicing requests or failed/timed-out requests.
Innovation Solution
Implementing a computer-implemented method that controls access to services by using bulkheads and shared burst bulkheads to manage concurrent active requests, routing requests through a shared burst bulkhead when the first bulkhead is maxed out, and refusing access when both bulkheads are at maximum capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If a single-threaded process architecture is used, then context switching overhead is eliminated, but latency increases due to blocking on input/output operations
Solution Approach 1:
The patent applies bulkhead segmentation by dividing the request handling capacity into separate bulkheads (first bulkhead with capacity N and second bulkhead with capacity M). This segmentation allows independent management of request flows, enabling the system to isolate blocking I/O operations within specific bulkheads while maintaining overall system responsiveness and reducing latency without the need for context switching.
2Loss of time
If bulkhead capacity is increased to reduce blocking delays, then request servicing improves, but system resources are consumed more quickly
Solution Approach 1:
The patent implements partial action by providing a first bulkhead with capacity N and a second bulkhead with capacity M, where N and M are configured to balance resource consumption with request handling capability. This partial capacity allocation allows the system to handle requests efficiently without over-provisioning resources, optimizing the trade-off between reducing blocking delays and conserving system resources.
3Productivity
If a multi-threaded architecture is used, then request throughput increases, but context switching between threads becomes computationally expensive
Solution Approach 1:
The patent introduces bulkheads as intermediary structures that mediate between request sources and database services. These bulkheads act as single-threaded event loops that handle requests sequentially without context switching, while still enabling high throughput by managing multiple request queues simultaneously. This intermediary approach eliminates the computational cost of thread context switching while maintaining request handling capability.
Data Source
AI summary
Systems and methods for controlling access to services. Methods may comprise receiving, from a first client of a plurality of clients, a first request to access a first service of a plurality of services. The first service may be associated with a first bulkhead. A first count of concurrent active requests to the first service via the first bulkhead may be determined. If the first count is equal to a first bulkhead maximum value, access to the first service via the first bulkhead may consequently be refused. A second count of concurrent active requests via a shared burst bulkhead may be determined. The second count may correspond to concurrent active requests to any of the plurality of services via the shared burst bulkhead. If the second count is less than a shared burst maximum value, the first request to the first service may be routed via the shared burst bulkhead.