Proxy Server Streaming Resources to Multiple Clients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content delivery networks (CDNs) face inefficiencies when handling large resources, as they often require waiting for the entire file to be received before streaming, leading to increased latency and bandwidth usage on the origin server, especially when multiple requests are made for the same resource before it is fully cached.
Innovation Solution
A proxy server with a multi-process architecture streams resources to multiple client devices while the file is being received, using a temporary cache file and shared memory to manage metadata, allowing simultaneous streaming without waiting for the file to be fully received or for a cache lock to expire.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the CDN server waits for the entire resource to be received before streaming to clients, then data integrity is ensured, but latency increases and origin server bandwidth is wasted
Solution Approach 1:
The patent applies preliminary action by starting to stream the resource to the first client immediately after receiving it from the origin server, before the entire resource is fully received and cached. This allows the first client to receive the resource without waiting for complete reception, reducing latency while maintaining data integrity through the caching mechanism.
Solution Approach 2:
The patent applies partial action by streaming only the portion of the resource that has been received so far to the first client, rather than waiting for the complete resource. This partial streaming approach reduces latency for the first client while the remaining portion continues to be received and cached in the background.
2Reliability
If the CDN server sets a cache lock to wait for the entire resource before handling subsequent requests, then data consistency is maintained, but origin server bandwidth is increased due to additional requests
Solution Approach 1:
The patent applies preliminary action by pre-establishing a cache lock mechanism that allows subsequent clients to wait in the cache for the resource without generating additional requests to the origin server. When the resource is received, all waiting clients can be served from the cache, preventing multiple origin server requests and conserving bandwidth.
Solution Approach 2:
The patent uses the cache as an intermediary mechanism between the origin server and multiple clients. The cache lock acts as a mediator that coordinates access to the resource being received, allowing multiple clients to be served from the cache once the resource is available, rather than generating multiple requests to the origin server.
3Speed
If the CDN server handles multiple requests for the same resource concurrently from different worker processes, then client request responsiveness is improved, but resource duplication and origin bandwidth usage increase
Solution Approach 1:
The patent applies merging by combining multiple client requests for the same resource into a single cache lock mechanism. Different worker processes can simultaneously register their requests in the shared cache lock structure, and when the resource is received, all waiting clients are served from the same cached copy, preventing multiple origin server requests.
Solution Approach 2:
The patent applies universality by creating a cache lock mechanism that serves multiple worker processes and clients simultaneously. The shared cache lock structure allows any worker process handling a request for the same resource to access the same caching mechanism, enabling universal resource sharing across multiple concurrent requests.
4Productivity
If the CDN server uses a multi-process architecture with independent worker processes, then processing parallelism is improved, but coordination overhead and memory sharing complexity increase
Solution Approach 1:
The patent uses the shared cache lock as an intermediary mechanism that simplifies coordination between independent worker processes. Instead of requiring complex inter-process communication protocols, worker processes can independently access the shared cache lock structure to register requests and retrieve resources, reducing coordination overhead while maintaining parallelism.
Solution Approach 2:
The patent extracts the coordination functionality from the worker processes themselves and places it in the shared cache lock mechanism. This separation allows worker processes to remain independent and simple, while the cache lock handles the coordination logic for resource sharing and request management across processes.
Data Source
AI summary
A first server receives a request for a file from a first client device. The first server determines that the file is not fully available in a cache that is available to the first server and receives at least a portion of the file from a second server. While the file is being received, the first server writes data of the file to a temporary cache file as it is being received; streams the file to the first client device from the temporary cache file; receives a request for the file from a second client device; and streams the file to the second client device from the temporary cache file without waiting for the requested file to be fully received at the first server and without transmitting an additional request for the file to the second server.