Database Query Optimization via IN-List to Join Transformation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Relational database systems face inefficiencies in accessing large tables due to the need for scanning all rows, which can impose a significant load on computing resources, especially when dealing with millions or billions of records.

Innovation Solution

The optimization technique transforms IN-List queries into join operations, allowing for the use of indexes instead of scan operations, thereby reducing the need to scan entire tables and optimizing access paths by evaluating the cost of different join methods and selecting the least costly access path.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If an all-row scan operation is used to access a large table, then all data can be retrieved, but the CPU usage and I/O operations increase significantly

Engineering Contradiction:
Improvedata retrieval completenessVSAvoidquery execution efficiency
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments the IN-List query into multiple individual equality conditions (e.g., column = value1 OR column = value2 OR column = value3) and processes each segment separately through index access paths, rather than scanning the entire table. This segmentation allows the optimizer to use efficient index seeks for each value and combine results, dramatically reducing I/O operations while maintaining complete data retrieval.

Inventive Principle:
Principle #1Segmentation

2Productivity

If an index is used for table access, then access efficiency improves, but an index key with constant values must be specified in the query

Engineering Contradiction:
Improvetable access efficiencyVSAvoidquery formulation complexity
Core Design Contradiction:
ProductivityVSEase of operation

Solution Approach 1:

The patent introduces an intermediary transformation process that converts the IN-List syntax (column IN (value1, value2, value3)) into an equivalent form that can utilize indexes. The optimizer acts as an intermediary by automatically rewriting the query to use individual equality comparisons with the indexed column, allowing index access without requiring the user to manually specify complex index keys. This intermediary transformation maintains ease of operation while achieving efficient index-based access.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Productivity

If the optimizer selects an access path for table access, then query execution is optimized, but scanning large tables still imposes unacceptable load on computing resources

Engineering Contradiction:
Improvequery execution speedVSAvoidcomputing resource consumption
Core Design Contradiction:
ProductivityVSLoss of energy

Solution Approach 1:

The patent applies preliminary action by having the optimizer evaluate and select the most efficient access path before executing the query. The system pre-assesses available indexes, estimates the cost of different access methods (scan vs. index seek), and chooses the optimal path in advance. This preliminary optimization prevents unnecessary full table scans and reduces computing resource consumption by using index seeks that access only the required rows based on the IN-List values.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS7882101B2Optimizing access to a database using in list queries
Publication Date: 2011.02.01 TERADATA US INC
  • US7882101B2 patent drawing
  • US7882101B2 patent drawing
  • US7882101B2 patent drawing

AI summary

A SQL query that includes an IN-List is optimized by (1) performing an evaluation to determine whether access to a table can be performed as a join operation, (2) converting the IN-List to an IN-LIST relation, and (3) joining the IN-List relation with the table to access the data in the table.