Cross-Column Relationship Detection Using Deep Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently detecting relationships across database columns, leading to suboptimal data storage and retrieval efficiency in data storage systems.
Innovation Solution
The use of feature-based and deep-learning-based similarity models to determine weighted similarity scores for tagged data columns, enabling the identification of related subsets and facilitating database consolidation operations across multiple databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to detect relationships across database columns, then the detection process is simple, but the detection accuracy and completeness of data relationships deteriorates
Solution Approach 1:
The patent segments the relationship detection process into multiple independent similarity measure components (semantic similarity, syntactic similarity, structural similarity, etc.), each handling specific aspects of column relationship detection. This segmentation improves detection accuracy by addressing different relationship dimensions separately while keeping each component manageable in complexity.
Solution Approach 2:
The patent introduces similarity measures as intermediary components that bridge the gap between raw column data and relationship detection results. These similarity measures act as mediators that transform complex column comparisons into quantifiable relationship scores, improving detection accuracy without requiring direct complex analysis of all column attributes.
2Measurement precision
If multiple similarity measures are combined to improve relationship detection, then the detection accuracy improves, but the computational complexity increases
Solution Approach 1:
The patent implements partial action by allowing users to selectively enable or disable specific similarity measure types based on their needs. Not all similarity measures are applied simultaneously in all cases - the system can adjust the extent of analysis to balance accuracy requirements against available computational resources.
Solution Approach 2:
The patent enables dynamic adjustment of similarity measure parameters and weights to optimize the balance between detection accuracy and computational cost. By changing parameters such as similarity thresholds, weights for different measure types, and selection of which measures to apply, the system can adapt to different performance and resource constraints.
3Manufacturing precision
If feature-based similarity models are used to determine weighted similarity scores, then the identification of related data columns improves, but the processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing feature representations of data columns, including semantic features, syntactic features, and structural features. These pre-computed features are cached and reused during relationship detection, eliminating the need to recalculate them each time and significantly reducing processing time while maintaining identification precision.
Solution Approach 2:
The patent extracts and separates the computationally intensive feature extraction process from the relationship detection process. By taking out the feature computation as a distinct preliminary step and storing results separately, the system avoids redundant calculations during detection and reduces overall processing time.
Data Source
AI summary
There is a need for more effective and efficient detection of cross-data-column relationships. This need can be addressed by, for example, techniques for detecting cross-data-column data relationships that utilize at least one of feature-based similarity models and deep-learning-based similarity models. The cross-data-column data relationships may be displayed to an end-user using a cross-column relationship detection user interface.