Main Memory Partitioning for Parallel Database Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient execution of queries and data processing.
Innovation Solution
A large-scale database system architecture that utilizes parallelized resources across multiple computing devices, enabling distributed query execution and optimized data storage through parallelized data input, storage, retrieval, and processing subsystems, along with a query and response system that optimizes query plans based on available resources and storage instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If database systems use traditional hardware and storage methods, then system complexity is low, but processing speed and query execution time are limited
Solution Approach 1:
The patent segments the database system into multiple computing devices, each handling specific portions of data and queries. This distributed architecture divides the monolithic system into manageable units that can process operations concurrently, thereby increasing processing speed while maintaining controlled complexity through modular design.
Solution Approach 2:
The patent transitions from a single-dimensional centralized processing model to a multi-dimensional distributed architecture across multiple computing devices. This dimensional expansion allows parallel processing of queries and data operations, significantly improving processing speed by utilizing multiple processing dimensions simultaneously.
2Loss of time
If database systems process queries sequentially, then resource allocation is simple, but query execution time increases
Solution Approach 1:
The patent implements continuous query processing across multiple computing devices where operations do not wait for sequential completion. Multiple queries are processed concurrently in parallel streams, eliminating idle time and ensuring continuous useful action throughout the system, thereby reducing overall query execution time.
Solution Approach 2:
The patent performs preliminary actions by pre-processing and caching data in memory buffers before queries are executed. This preliminary preparation of data in accessible memory formats reduces the time required for actual query execution, as data is already positioned and formatted for rapid retrieval and processing.
3Productivity
If database systems use centralized data storage, then data access is simplified, but processing scalability is restricted
Solution Approach 1:
The patent segments centralized data storage into distributed storage across multiple computing devices. Each device maintains portions of the database in its local memory and storage, allowing independent access and processing. This segmentation enables linear scalability as new devices can be added to the distributed architecture without modifying the core storage structure.
Solution Approach 2:
The patent creates a universal distributed storage architecture where each computing device can serve multiple functions: storing data locally, processing queries against local data, and participating in distributed query execution. This multi-functionality enhances processing scalability as each device contributes to both storage capacity and processing power simultaneously.
4Speed
If database systems allocate memory for each computing device, then data access speed improves, but memory management complexity increases
Solution Approach 1:
The patent implements self-service memory management where each computing device autonomously manages its own allocated memory resources. Devices independently allocate and manage their memory buffers, cache structures, and data storage without requiring centralized memory control, thereby improving data access speed through local memory operations while keeping memory management complexity distributed and manageable.
Solution Approach 2:
The patent allocates excessive memory resources to each computing device beyond strict minimum requirements, allowing devices to pre-load and cache data proactively. This partial over-allocation of memory enables faster data access through local caching, while the distributed nature of memory management across multiple devices prevents any single device from becoming a memory management bottleneck.
Data Source
AI summary
A node of a computing system includes a main memory and a plurality of processing core resources. The main memory includes a computing device section and a database section. The computing device section includes a computing device operating system area and a computing device general area. The database section includes a database section that includes a database operating system area, a disk area, a network area, and a database general area. The database operating system area allocates at least one portion of the main memory for database operations that is locked from access by the computing device operating system area.


