Dynamic Network Buffer Sizing for Database Data Transfers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Static buffer sizing in network layers of database systems leads to suboptimal performance during data transfers, resulting in either prolonged data transfer times due to large buffers or increased system resource consumption due to frequent system calls with small buffers.
Innovation Solution
Dynamically adjusting the network buffer size based on the type and size of database calls in progress, allowing for optimal buffer allocation for each operation, thereby reducing system call overhead and enhancing throughput performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the network buffer size is configured to be a fixed size statically, then the system configuration is simple and easy to manage, but the network buffer may not be suitable for at least some of the data transfers between database systems, resulting in suboptimal performance
Solution Approach 1:
The patent implements dynamic buffer sizing where the network buffer size is automatically adjusted based on the characteristics of each database call. The system determines buffer size dynamically according to the type of database call in progress and the data size used by the call, allowing the buffer to adapt to different transfer scenarios rather than using a fixed static size configuration
Solution Approach 2:
The system changes the buffer size parameter based on the specific requirements of each database operation. By monitoring the type of database call and the amount of data to be transferred, the system modifies the buffer size parameter in real-time to optimize performance for each specific transfer scenario
2Quantity of substance
If the buffer size is too large, then the buffer can hold more data, but the data may not be transferred to the system at the other end for a long duration, resulting in unacceptably low throughput performance
Solution Approach 1:
The system dynamically adjusts buffer size to match the actual data transfer requirements. Instead of using a consistently large buffer, the buffer size is scaled up or down based on the specific database call and data size, ensuring optimal throughput for each transfer operation while avoiding the performance degradation associated with overly large buffers
3Quantity of substance
If the buffer size is too small, then the buffer uses less memory, but the buffer may need to be filled multiple times to effect a large data transfer, and system calls consume substantial system resources, affecting overall throughput performance
Solution Approach 1:
The system dynamically determines the appropriate buffer size based on the data size and type of database call. For large data transfers, the buffer size is increased to reduce the number of system calls needed, while for smaller transfers, a smaller buffer is used to conserve memory. This dynamic adjustment optimizes the balance between memory usage and throughput performance
4Ease of operation
If system calls are used to place data into the buffer and for the network layer to transfer the data, then data transfer can be effected, but system calls consume substantial system resources, affecting overall throughput performance
Solution Approach 1:
The system dynamically optimizes the number and size of system calls by adjusting the buffer size to match the data transfer requirements. By using appropriately sized buffers, the system reduces the frequency of system calls for large data transfers, thereby reducing system resource consumption while maintaining the necessary data transfer capability
Data Source
AI summary
In transferring data from/to a database server, the size of a network buffer used by a network layer is dynamically adjusted according to the call type and expected size of data to be transferred for the call. Due to such use of appropriate buffer size, throughput performance of the transfer may be enhanced.