Direct Table Association for In-Memory Database Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database architectures experience high latency due to inefficient query processing, particularly in high concurrency scenarios, as they rely on disk-based storage which slows down data retrieval.
Innovation Solution
Implementing an in-memory database management system that establishes direct table associations using physical memory address pointers, allowing for faster data access and reducing the need for repetitive scanning during join operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If disk-based storage is used to store database data, then data persistence is ensured, but data access latency increases significantly
Solution Approach 1:
The patent creates in-memory copies of database tables and maintains direct table associations in RAM, allowing frequent access operations to occur on these copies rather than accessing the original disk-based storage directly. This copying approach enables fast data retrieval while the original disk-based storage maintains data persistence.
Solution Approach 2:
The system pre-loads table data into memory and pre-establishes direct table associations between related tables during database initialization or data loading phases. This preliminary action ensures that when queries are executed, the data and associations are already in memory, eliminating the need for repeated disk access during query processing.
2Reliability
If traditional join operations are performed by scanning tables, then all possible matches are found, but processing time increases in high concurrency scenarios
Solution Approach 1:
The patent pre-establishes direct table associations that store pointers or references from foreign key columns to primary key columns of related tables. This preliminary structuring of data relationships allows the system to directly navigate to matching records without performing full table scans during query execution, thereby maintaining result completeness while dramatically improving processing speed.
Solution Approach 2:
The system replaces the mechanical scanning process (sequentially reading through table rows to find matches) with a direct pointer-based navigation mechanism. Instead of mechanically scanning tables to find related records, the system uses pre-computed memory addresses and pointers to directly access matching records, substituting a slow mechanical process with a fast direct access mechanism.
3Speed
If in-memory database is implemented, then data access speed increases, but memory consumption increases
Solution Approach 1:
The patent divides the database into separate table structures, each with its own direct association data structure. By segmenting the in-memory database into discrete table components rather than maintaining a single large monolithic structure, the system can more efficiently manage memory allocation and potentially evict less frequently accessed tables from memory while keeping frequently accessed tables in RAM.
Data Source
Figure 1
Figure 2A~2B
Figure 3A~3B
AI summary
To address the issue of latency in relational database systems, a computerized database management system comprising a database server, a processor, and associated random access memory is provided. The processor may configure an in-memory database in the random access memory to have tables, each containing a plurality of cells with associated memory addresses, establish a direct table association between an origin cell and a target cell within the tables, receive a request to perform a join database operation of a first table including the origin cell on a second table including the target cell, dereference the direct table association for the origin cell to obtain a value of the target cell stored at the memory address for the target cell, execute the join database operation to produce a joined table including the value, and store the joined table at a new location in the random access memory.