Divided Main Memory for Database and Computing Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient execution of complex database functions.
Innovation Solution
A database management system employing a parallelized architecture with a hybrid indexing and hierarchical query processing approach, utilizing multiple nodes and processing core resources to manage data partitions, optimize query plans, and facilitate lock-free and parallel execution of operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a traditional single-memory architecture is used, then device complexity is low, but processing speed and productivity are limited
Solution Approach 1:
The main memory is divided into two distinct sections: a computing device memory section and a database memory section. This segmentation allows independent optimization of each section for its specific workload, enabling parallel processing and improving overall system productivity without requiring complete system redesign
Solution Approach 2:
The patent introduces a new dimensional organization of memory by creating separate address spaces for computing device threads and database threads. This dimensional separation enables concurrent memory access without interference, significantly improving processing speed while maintaining manageable complexity through structured organization
2Productivity
If main memory is not divided into sections, then device complexity is low, but threads cannot access memory independently and processing efficiency is reduced
Solution Approach 1:
The memory is segmented into computing device memory section and database memory section with independent access rights. Computing device threads access the computing device memory section while database threads access the database memory section, enabling independent parallel processing and improving thread processing efficiency
Solution Approach 2:
Each memory section is optimized for its specific access pattern: the computing device memory section is optimized for general-purpose computing operations while the database memory section is optimized for structured data access. This local optimization improves processing efficiency without requiring global memory redesign
3Speed
If hybrid indexing is not used, then device complexity is low, but query processing speed is limited
Solution Approach 1:
The patent merges multiple indexing techniques into a hybrid indexing structure that combines traditional B-tree indexes with hash indexes and inverted indexes. This combination allows the system to leverage the strengths of each indexing method for different query types, significantly improving query processing speed
Solution Approach 2:
The hybrid indexing structure provides multi-functionality by supporting various query patterns (exact match, range queries, full-text search) through different index types within the same database system. This universal approach improves query processing speed across diverse workloads without requiring separate indexing solutions
4Loss of time
If parallel execution is not enabled, then device complexity is low, but processing time increases
Solution Approach 1:
The memory segmentation into computing device and database sections enables independent parallel execution of operations in each section. Multiple threads can simultaneously access different memory sections without conflict, reducing processing time while managing complexity through structured concurrency
Solution Approach 2:
The system performs preliminary actions by pre-loading data into appropriate memory sections and pre-compiling query plans before actual processing. This preparation enables faster parallel execution during query processing, reducing overall processing time without adding significant complexity to the execution engine
Data Source
AI summary
A computing device includes a plurality of nodes, wherein a first node of the plurality of nodes operates in accordance with a computing device operation system (OS) and remaining nodes of the plurality of nodes operate in accordance with a database OS and process a plurality of threads of an application. The computing device further includes a divided main memory that is divided into a computing device memory section and a database memory section, and the database OS determines an allocation of the divided main memory between the computing device memory section and the database memory section, where a first database thread is assigned a buffer of a plurality of buffers of the database memory section, and a first computing device thread utilizes the computing device memory section of the divided main memory.