Holistic Cross-Data-Source Linking in Data Warehouses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data warehouse systems face inefficiencies in cross-data-source linking due to non-holistic approaches, leading to performance degradations and time-consuming data processing, which hinders timely and real-time reporting in systems like flight management.
Innovation Solution
Implementing a holistic cross-data-source linking method by importing new data entries into a staging area, identifying backbone and non-backbone entries, and iteratively linking them to reduce time complexity and resource usage, utilizing conceptual linking patterns between data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional non-holistic cross-data-source linking is used, then data processing can be performed, but time-intensiveness increases and processor utilization deteriorates
Solution Approach 1:
The patent segments the cross-data-source linking process into distinct phases: identifying backbone data sources, establishing backbone-to-non-backbone links, and processing non-backbone data sources. This segmentation allows the system to focus computational resources on critical linking operations first, reducing overall time-intensiveness while maintaining complete data integration.
Solution Approach 2:
The patent performs preliminary action by pre-identifying backbone data sources and establishing their linking relationships before processing non-backbone data sources. This preliminary structuring of the data linkage framework enables subsequent non-backbone sources to be integrated more efficiently, reducing total processing time and improving processor utilization.
2Productivity
If traditional cross-data-source linking is used, then data integration is achieved, but processor utilization deteriorates
Solution Approach 1:
The patent segments processor workload by separating backbone data source processing from non-backbone data source processing. This segmentation reduces peak processor utilization by distributing computational tasks across different phases, preventing resource bottlenecks while achieving complete data integration.
Solution Approach 2:
The patent applies partial action by initially focusing processing resources exclusively on backbone data sources and their direct relationships, then progressively incorporating non-backbone data sources. This staged approach reduces immediate processor utilization demands while ultimately achieving full data integration.
3Loss of time
If holistic cross-data-source linking is implemented, then time-intensiveness is reduced, but device complexity increases
Solution Approach 1:
The patent manages device complexity by segmenting the holistic linking system into modular components: backbone identification modules, linking relationship establishment modules, and data source integration modules. This modular segmentation reduces overall system complexity while enabling efficient time-intensive processing through coordinated operation of specialized components.
Data Source
AI summary
Method, apparatus and computer program product for linking data entries across data sources. For example, the apparatus includes at least one processor and at least one non-transitory memory including program code. The at least one non-transitory memory and the program code are configured to, with the at least one processor, store unlinked data entries in a staging memory area; store linked data entries in an active memory area; identifying a linked state status for the staging memory area, wherein the linked state status initially indicates a non-linked state; repeatedly performing one or more cross-data-source linking operations until the linked state status for the staging memory area indicates a linked state; and in response to determining that the linked state status for the staging memory area indicates the linked state, linking the multiple data entries by merging the staging memory area and the active memory area to generate linked data.