Database Normalization via Statistical Cost Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transforming a denormalized database into a normalized schema is challenging due to the complexity of identifying and maintaining functional dependencies, potential errors in data, and the existence of multiple decompositions that satisfy normal forms, leading to inefficient and suboptimal schema selection.
Innovation Solution
A method is introduced that constructs a snowflake schema by decomposing the denormalized database into related tables connected by foreign keys, using a cost model that balances the storage cost of tables and foreign keys, and dynamically determines the number of tables based on entropy values, thereby selecting an optimal schema for storage compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If a denormalized table is decomposed into multiple normalized schemas, then data redundancy is reduced and storage efficiency is improved, but the complexity of identifying and maintaining functional dependencies increases and the process becomes time-consuming
Solution Approach 1:
The patent replaces manual or rule-based schema decomposition with a machine learning model that automatically learns functional dependencies from data patterns. The ML model analyzes the denormalized table and generates normalized schemas without requiring explicit specification of dependencies by users, thereby reducing the complexity of identifying and maintaining functional dependencies while still achieving data redundancy reduction.
Solution Approach 2:
The system enables self-service normalization where the ML model autonomously performs schema decomposition by learning directly from the data. The model automatically identifies patterns, determines functional dependencies, and generates optimal normalized schemas without external intervention, making the process efficient and adaptive to different data types.
2Manufacturing precision
If multiple possible schemas are considered for normalization, then the quality of the selected schema can be improved, but the time required for schema selection increases
Solution Approach 1:
The patent performs preliminary actions by pre-training the ML model on various normalization patterns and functional dependency relationships. This pre-training enables the model to quickly evaluate multiple schema options and select the optimal one during actual use, thereby improving schema quality without significantly increasing selection time for new datasets.
Solution Approach 2:
The system changes parameters by using probabilistic scoring and confidence thresholds to evaluate schema options. The ML model generates multiple candidate schemas with associated quality metrics, and the optimal schema is selected based on these parameters, allowing for efficient comparison and selection without exhaustive evaluation of all possible schemas.
3Ease of manufacture
If a normalized schema is generated without considering storage cost, then the normalization process is simpler, but the storage efficiency and file size are not optimized
Solution Approach 1:
The patent incorporates feedback loops where the ML model evaluates both the structural quality of generated schemas and their storage efficiency implications. The model adjusts its decomposition strategy based on feedback about storage costs, learning from patterns that lead to both high normalization quality and optimal storage utilization, thereby achieving both simplicity and storage optimization.
Data Source
AI summary
A method for generating schemas to compress storage of denormalized data includes receiving user selection of an initial data table and constructing a snowflake schema to decompose the initial data table into a plurality of tables. Tables in the plurality of tables that are related to one another are connected by foreign keys. The snowflake schema is constructed according to a cost model that balances the cost of storing the plurality of tables and the cost of storing the foreign keys. The method also includes transforming data in the initial data table into the plurality of tables according to the snowflake schema.


