Functional Dependency Discovery via Database Sampling and Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for discovering functional dependencies in large databases are inefficient due to high computational effort and memory requirements, especially as database sizes grow, with existing algorithms either requiring excessive resources or being inadequate for very large relations.

Innovation Solution

The approach involves sampling data from a relation to identify candidate functional dependencies, verifying these dependencies within the sample, and using pruning and database-statement-based verification to efficiently determine whether they hold for the entire relation, thereby reducing the search space and optimizing resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If existing algorithms are used to discover functional dependencies in large databases, then completeness of dependency discovery is improved, but computational effort and memory requirements increase excessively

Engineering Contradiction:

Improvecompleteness of dependency discoveryVSAvoidcomputational effort

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments the database into multiple blocks and processes each block separately to discover functional dependencies. Instead of loading the entire database into memory, the system divides it into manageable segments that can be processed in chunks, significantly reducing memory requirements while maintaining comprehensive dependency discovery across the entire database.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary indexing and data preparation operations before the actual functional dependency discovery process. By pre-processing the database to create efficient data structures and indexes, the system reduces the computational effort required during the main dependency discovery phase, enabling scalable processing of large databases.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If existing algorithms are used to discover functional dependencies in large databases, then accuracy of dependency identification is improved, but memory requirements increase excessively

Engineering Contradiction:

Improveaccuracy of dependency identificationVSAvoidmemory requirements

Core Design Contradiction:

Measurement precisionVSVolume of stationary object

Solution Approach 1:

The patent divides the database into multiple blocks that can be processed independently in memory. By processing one block at a time and using efficient data structures for each block, the system maintains high accuracy in dependency identification while keeping memory usage within practical limits, even for very large databases.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent uses sampling techniques where a representative subset (sample) of the database is copied and processed to identify candidate functional dependencies. This sampling approach allows accurate dependency discovery with minimal memory requirements, as only a fraction of the total database needs to be loaded into memory at any given time.

Inventive Principle:

Principle #26Copying

3Productivity

If sampling is used to discover functional dependencies, then computational efficiency is improved, but measurement precision of dependency discovery may be reduced

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidaccuracy of dependency discovery

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements a feedback mechanism where initial sampling results are used to guide subsequent processing. Candidate dependencies identified through sampling are verified against additional data blocks, and the system continuously refines its results based on feedback from each processing stage, ensuring high accuracy while maintaining computational efficiency.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent performs preliminary sampling to identify candidate functional dependencies quickly, then uses these candidates to guide more thorough verification processes. This two-stage approach with preliminary action allows the system to achieve both computational efficiency and high accuracy by focusing detailed analysis only on promising candidates rather than examining all possible dependencies.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS7873628B2Discovering functional dependencies by sampling relations

Publication Date: 2011.01.18 ORACLE INT CORP

AI summary

To discover functional dependencies in a large relation, a sample of tuples from the relation is collected. The sample is examined to determine whether one or more candidate functional dependencies exist just within the sample as a nominal dependency. When a nominal dependency is found in the sample, than all the tuples in the relation are examined to verify whether the nominal dependency holds for the whole relation. Candidate functional dependencies are disqualified when either a nominal dependency is found or when it is verified as functional dependency that holds for the entire relation.