Database Normalization via Statistical Cost Model

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Transforming a denormalized database into a normalized schema is challenging due to the complexity of identifying and maintaining functional dependencies, potential errors in data, and the existence of multiple decompositions that satisfy normal forms, leading to inefficient and suboptimal schema selection.

Innovation Solution

A method is introduced that constructs a snowflake schema by decomposing the denormalized database into related tables connected by foreign keys, using a cost model that balances the storage cost of tables and foreign keys, and dynamically determines the number of tables based on entropy values, thereby selecting an optimal schema for storage compression.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of substance

If a denormalized table is decomposed into multiple normalized schemas, then data redundancy is reduced and storage efficiency is improved, but the complexity of identifying and maintaining functional dependencies increases and the process becomes time-consuming

Engineering Contradiction:
Improvedata redundancyVSAvoidschema decomposition complexity
Core Design Contradiction:
Loss of substanceVSDevice complexity

Solution Approach 1:

The patent replaces manual or rule-based schema decomposition with a machine learning model that automatically learns functional dependencies from data patterns. The ML model analyzes the denormalized table and generates normalized schemas without requiring explicit specification of dependencies by users, thereby reducing the complexity of identifying and maintaining functional dependencies while still achieving data redundancy reduction.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service normalization where the ML model autonomously performs schema decomposition by learning directly from the data. The model automatically identifies patterns, determines functional dependencies, and generates optimal normalized schemas without external intervention, making the process efficient and adaptive to different data types.

Inventive Principle:
Principle #25Self-service

2Manufacturing precision

If multiple possible schemas are considered for normalization, then the quality of the selected schema can be improved, but the time required for schema selection increases

Engineering Contradiction:
Improveschema qualityVSAvoidschema selection time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-training the ML model on various normalization patterns and functional dependency relationships. This pre-training enables the model to quickly evaluate multiple schema options and select the optimal one during actual use, thereby improving schema quality without significantly increasing selection time for new datasets.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by using probabilistic scoring and confidence thresholds to evaluate schema options. The ML model generates multiple candidate schemas with associated quality metrics, and the optimal schema is selected based on these parameters, allowing for efficient comparison and selection without exhaustive evaluation of all possible schemas.

Inventive Principle:
Principle #35Parameter changes

3Ease of manufacture

If a normalized schema is generated without considering storage cost, then the normalization process is simpler, but the storage efficiency and file size are not optimized

Engineering Contradiction:
Improvenormalization process simplicityVSAvoidstorage efficiency
Core Design Contradiction:
Ease of manufactureVSLoss of substance

Solution Approach 1:

The patent incorporates feedback loops where the ML model evaluates both the structural quality of generated schemas and their storage efficiency implications. The model adjusts its decomposition strategy based on feedback about storage costs, learning from patterns that lead to both high normalization quality and optimal storage utilization, thereby achieving both simplicity and storage optimization.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS12153560B1Database normalization using statistical analysis
Publication Date: 2024.11.26 TABLEAU SOFTWARE INC
  • US12153560B1 patent drawing
  • US12153560B1 patent drawing
  • US12153560B1 patent drawing

AI summary

A method for generating schemas to compress storage of denormalized data includes receiving user selection of an initial data table and constructing a snowflake schema to decompose the initial data table into a plurality of tables. Tables in the plurality of tables that are related to one another are connected by foreign keys. The snowflake schema is constructed according to a cost model that balances the cost of storing the plurality of tables and the cost of storing the foreign keys. The method also includes transforming data in the initial data table into the plurality of tables according to the snowflake schema.