Database Server Column Data Transmission Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Column-oriented database systems face challenges in preventing increased network traffic between servers and clients due to the transmission of large value lists, particularly when using the FAST structure, which can lead to inefficient data transmission and network congestion.
Innovation Solution
The database system employs a data transmission mechanism that calculates transfer costs for different data structures for each column of the execution result and selects the most efficient structure for transmission, allowing for optimized data transfer by using either value number/value list or value array structures based on calculated costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transmitted using the FAST structure to secure data consistency, then data consistency is improved, but network traffic increases
Solution Approach 1:
The patent segments the data transmission process by separating the value list transmission from the actual data retrieval. Instead of transmitting the entire FAST structure value list, the system divides transmission into two phases: first transmitting a reference to the value list, then transmitting only the specific data values needed based on index values. This segmentation reduces network traffic while maintaining data consistency.
Solution Approach 2:
The patent extracts only the necessary data elements from the FAST structure for transmission. By using index values to reference specific positions in the value list, the system extracts and transmits only the required data values rather than the entire value list, thereby reducing network traffic while preserving data consistency through the index-based reference mechanism.
2Quantity of substance
If data is converted from FAST structure to line-oriented data structure, then network traffic is reduced, but duplicate data transmission occurs
Solution Approach 1:
The patent introduces a dynamic data structure that can adapt between FAST structure and line-oriented structure based on transmission needs. The system dynamically selects which columns to transmit in which structure format, using the index value mechanism to maintain consistency while allowing flexible structure conversion. This dynamic approach reduces duplicate data transmission by intelligently selecting the optimal structure for each column.
3Device complexity
If the same data structure is used for all columns, then system complexity is reduced, but data transfer efficiency decreases
Solution Approach 1:
The patent applies local quality by allowing different data structures to be used for different columns based on their specific characteristics. Each column can be evaluated independently, and the optimal structure (FAST or line-oriented) is selected for each column based on factors like data type, size, and access patterns. This localized optimization improves overall data transfer efficiency while maintaining manageable system complexity through standardized processing logic.
Data Source
AI summary
A database system comprising: a client that executes a query to a database server; and the database server that, according to a query from the client, transmits a table of an execution result of the query, the table representing an answer to each query as a record and including the record; wherein the database server comprises a data transmission part which, in a case of transmission of the table of the execution result having a plurality of columns, performs the transmission for each column of the table of the execution result by using any of a plurality of data structures being capable of representing the column.