Database Query Processing with RID List Merging

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Database applications employing star or snowflake schema face challenges in efficient query processing due to the need for joining multiple dimension tables with a fact table, leading to increased memory consumption and performance issues from large indexes.

Innovation Solution

A system and method for automatic multidimensional query processing that semi-joins dimension tables with a fact table, generates and merges record identification (RID) lists, sorts them based on data location, and fetches data values from the fact table, optimizing query execution and reducing resource consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a Cartesian product of dimension tables is formed for matching against a multi-column fact table index, then independent dimension filtering is consolidated, but memory consumption increases and performance deteriorates due to large indexes

Engineering Contradiction:
Improvedimension filtering consolidationVSAvoidmemory consumption
Core Design Contradiction:
Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent segments the dimension filtering process by creating separate RID lists for each dimension table instead of forming a complete Cartesian product. Each RID list contains only the relevant record identifiers for that dimension, allowing independent filtering without materializing all possible combinations. This segmentation reduces memory consumption while maintaining the ability to consolidate filtering results through intersection operations on the RID lists.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If a Cartesian product of dimension tables is formed for matching against a multi-column fact table index, then independent dimension filtering is consolidated, but query processing performance deteriorates

Engineering Contradiction:
Improvedimension filtering consolidationVSAvoidquery processing performance
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent extracts only the necessary record identifiers (RIDs) from each dimension table that match the query criteria, rather than processing entire tables or their Cartesian product. By extracting and storing RIDs in separate lists, the system consolidates dimension filtering through intersection operations on these compact lists, significantly reducing the computational burden and improving query processing performance while maintaining filtering versatility.

Inventive Principle:
Principle #2Taking out (Extraction)

3Loss of substance

If dimension tables are normalized into separate tables (snowflake schema), then data redundancy is reduced, but the number of tables to be joined increases

Engineering Contradiction:
Improvedata redundancyVSAvoidnumber of joins
Core Design Contradiction:
Loss of substanceVSDevice complexity

Solution Approach 1:

The patent introduces RID lists as intermediary structures that mediate between normalized dimension tables and the fact table. Instead of performing multiple sequential joins between normalized dimension tables, the system generates RID lists from each dimension table independently, then uses these RID lists to directly access relevant records in the fact table. This intermediary approach maintains the benefits of normalization (reduced redundancy) while simplifying the join operation complexity.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11893020B2Enhancing database query processing
Publication Date: 2024.02.06 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US11893020B2 patent drawing
  • US11893020B2 patent drawing
  • US11893020B2 patent drawing

AI summary

A system, program product, and method for enhancing automatic multidimensional query processing. The method includes executing a database query including semi-joining a plurality of dimension tables with a fact table. The method also includes identifying for extraction one or more data values from each dimension table of the plurality of dimension tables. The data values from each dimension table of the plurality of dimension tables are associated with a respective record identification (RID), thereby defining one or more RIDs. The method further includes generating a plurality of RID lists. Each RID list of the plurality of RID lists includes a collection of the one or more RIDs for the respective dimension table. The method also includes merging the plurality of RID lists, sorting, subject to the merging, the plurality of RIDs as a function of data location, and fetching the data values from the fact table.