Adaptive Data Transfer Protocol Parallel Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data transfer protocols are limited by single network connections, leading to bottlenecks in network throughput and performance, especially in database queries, which cannot be fully addressed by upgrading hardware alone.
Innovation Solution
An adaptive data transfer protocol that manages multiple network connections within a logical connection, utilizing adaptive compression and load-balancing strategies based on workload metrics, and employing best-of-breed compression algorithms to improve data transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a data transfer protocol uses a single network connection for database queries, then the protocol structure remains simple and easy to manage, but network throughput becomes a bottleneck and performance is limited
Solution Approach 1:
The patent divides a single logical connection into multiple physical network connections (e.g., connection 0, connection 1, connection 2). Each physical connection can independently transmit data packets, allowing parallel data transfer and significantly increasing network throughput while maintaining a unified logical connection interface for applications.
Solution Approach 2:
The patent introduces the dimension of connection multiplicity by allowing multiple physical connections within a single logical connection. This transforms the traditional single-dimension connection model into a multi-dimensional structure where data can flow through multiple parallel paths simultaneously,突破ing the throughput limitations of single connections.
2Productivity
If multiple logical connections are used between clients and server, then network throughput can be increased, but connection management becomes complex and difficult to handle unpredictable database traffic
Solution Approach 1:
The patent creates a universal logical connection interface that can work with multiple physical connections simultaneously. The logical connection layer provides a unified management interface that handles routing, load balancing, and error recovery across multiple physical connections, making the system as easy to manage as a single connection while achieving multiple connection throughput.
Solution Approach 2:
The logical connection acts as an intermediary layer between applications and physical network connections. It mediates all communication by receiving data from applications, distributing it across multiple physical connections, and reassembling responses, thereby simplifying connection management for applications while enabling parallel data transfer.
3Productivity
If data is compressed for network transmission, then network efficiency is improved and bandwidth is reduced, but processing overhead increases and may degrade performance when compression is not beneficial
Solution Approach 1:
The patent implements dynamic compression that automatically adjusts compression settings based on data characteristics, connection type, and performance metrics. The system monitors compression effectiveness in real-time and adapts compression algorithms and intensity, enabling compression to be applied selectively when beneficial while avoiding unnecessary processing overhead when compression would not improve performance.
Solution Approach 2:
The patent changes compression parameters dynamically based on data types, connection bandwidth, and workload conditions. Different compression algorithms and intensity levels are applied to different data streams based on their characteristics, optimizing the balance between compression overhead and network efficiency gains for each specific scenario.
Data Source
AI summary
There is disclosed an adaptive data transfer protocol, system and method. In an embodiment, the method is operable on one or more computer devices in a client server data query environment, each of the one or more computer devices having a processor and a memory, the method comprising: opening a logical network connection between a client and a server, the server communicating with a data source; initiating a data request from the client; in response to the data request from the client, preparing a data transfer response from the server utilizing an adaptive data transfer protocol, the adaptive data transfer protocol configured to: create multiple network connections within the logical connection; analyze performance metrics including variable processor load on the one or more computing devices, and network load over the multiple network connections; analyze the data required to be retrieved from the data source and transferred over the multiple network connections; and adaptively prepare the data retrieved from the data source based on their data types and values for transfer over the multiple network connections to improve total response time to fulfill the data request.