Join Decision Manager for Broadcast vs Hash-Hash Join Selection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing database systems face challenges in efficiently executing join operations, particularly when the size of the build-side data is larger than the probe-side data, leading to inefficient resource utilization and increased processing time.

Innovation Solution

Implementing a join decision manager (JDM) that determines whether a join operation should be executed as a broadcast join or a hash-hash join based on the size of the build-side and probe-side data, reducing the use of expensive broadcast joins by opting for hash-hash joins when the probe-side data is not significantly larger than the build-side data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If broadcast join is used when build-side data is larger than probe-side data, then join operation can be completed, but resource utilization becomes inefficient and processing time increases

Engineering Contradiction:

Improvejoin execution efficiencyVSAvoidresource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system dynamically changes the join execution parameter (join type) based on data size parameters. The join decision manager evaluates the size relationship between build-side and probe-side data, then selects the appropriate join type (broadcast join or hash-hash join) to optimize resource utilization and execution efficiency.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The join execution strategy is made dynamic rather than static. The system adapts the join type based on runtime conditions (data sizes), allowing the execution plan to change according to actual data characteristics, thereby avoiding inefficient broadcast joins when they are not appropriate.

Inventive Principle:

Principle #15Dynamics

2Productivity

If broadcast join is used frequently, then all join operations can be executed, but processing time increases due to expensive broadcast operations

Engineering Contradiction:

Improvejoin processing speedVSAvoidjoin execution time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system changes the execution parameter (join type) based on the size parameter of the data. When probe-side data is not significantly larger than build-side data, the system switches from broadcast join to hash-hash join, reducing execution time while maintaining correctness.

Inventive Principle:

Principle #35Parameter changes

3Ease of operation

If join type selection is made without considering data size relationship, then execution is simple, but resource utilization becomes inefficient

Engineering Contradiction:

Improvejoin execution simplicityVSAvoidresource utilization efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The join decision manager automatically evaluates data sizes and selects the appropriate join type without requiring manual intervention. The system serves itself by making intelligent decisions based on runtime conditions, improving resource utilization while maintaining ease of operation for users.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12530354B1Probe-side annotated join decision making

Publication Date: 2026.01.20 SNOWFLAKE INC

AI summary

A join decision manager (JDM) generates a data processing pipeline. The data processing pipeline includes at least one join operation associated with build-side row data and probe-side row data. The JDM determines the maximum cardinality associated with the probe-side row data. The JDM determines size of the build-side row data at a decision node of the at least one join operation. The JDM configures execution of the at least one join operation as one of a broadcast join or a hash-hash join based on the size of the build-side row data and the maximum cardinality.