Partitioned Columnar Database Local Query Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer systems face significant delays in processing and loading websites due to connection latency, data transfer time, backbone latency, and application latency, which negatively impact user experience and can result in reduced traffic and revenue.
Innovation Solution
A computer-implemented query system that allows clients to receive and store partitions of a server database, enabling local execution of queries without network latency, using compression and encoding techniques to reduce data transfer times and improve response speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If queries are executed on remote servers, then data accuracy and completeness are maintained, but network latency and connection time significantly increase query execution time
Solution Approach 1:
The database is divided into partitions that can be distributed to client systems. Each partition contains a subset of the total data, allowing clients to execute queries locally on relevant partitions without needing to access the complete remote database, thereby reducing network latency while maintaining query accuracy.
Solution Approach 2:
Database partitions are pre-loaded and cached on client systems before queries are executed. This preliminary action stores frequently accessed data locally, enabling clients to perform queries immediately without waiting for network responses, thus significantly reducing query execution time.
2Productivity
If complete database partitions are transferred to clients, then local query execution speed improves, but data transfer time and network bandwidth consumption increase
Solution Approach 1:
Different parts of the database are selectively distributed to different client systems based on their specific needs and query patterns. Each client receives only the partition relevant to its function, optimizing local query execution speed while minimizing unnecessary data transfer and network bandwidth consumption.
3Quantity of substance
If data is compressed and encoded for transmission, then data transfer efficiency improves, but client-side processing complexity increases
Solution Approach 1:
Data compression and encoding are applied on the server side before transmission to clients. The server prepares the data in an optimized format, performing the complex processing in advance, which improves data transfer efficiency while keeping client-side processing relatively simple.
Data Source
AI summary
A client system can improve processing speeds by executing queries locally. A remote database can have a partitioned, columnar database. Queries from the client can be executed on the remote database, and blocks of a partition of the partitioned, columnar database can be sent to the client. Once the client has enough blocks to form the partition, the client can execute queries that reference the partition. The queries can be executed using templates for specialized expression evaluators.