Method and apparatus for acquiring bloodline data between heterogeneous data
By using distributed system child nodes and metadata extraction technology, the lineage relationship of heterogeneous data throughout its entire lifecycle is collected and constructed, which solves the shortcomings of existing technologies in the collection and analysis of multi-source heterogeneous data and improves the accuracy and coverage of data fusion.
CN116450624BActive Publication Date: 2026-06-19DIANKEYUN (BEIJING) TECH CO LTD +1
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DIANKEYUN (BEIJING) TECH CO LTD
- Filing Date
- 2023-02-13
- Publication Date
- 2026-06-19
AI Technical Summary
Technical Problem
Existing technologies cannot effectively collect and analyze the full lifecycle lineage of multi-source heterogeneous data, especially lacking coverage and accuracy in large-scale distributed systems.
Method used
The system employs child nodes in a distributed system to collect data throughout its entire lifecycle. Combined with metadata extraction technology, it constructs global lineage data. Through a heterogeneous data extraction module and a distributed metadata module, it performs deduplication and fusion to establish a lineage graph.
Benefits of technology
It enables precise collection and full lifecycle coverage of heterogeneous data, effectively constructs lineage relationships in distributed systems, and improves the traceability and accuracy of data fusion.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN116450624B_ABST
Abstract
This application provides a method and apparatus for obtaining lineage data between heterogeneous data. The method includes: collecting offline heterogeneous data and collecting online heterogeneous data based on metadata-based heterogeneous data extraction technology; using each sub-node in a distributed system to perform distributed metadata collection on the full lifecycle data of various types of data in the heterogeneous data; aggregating the distributed metadata to the master node in the distributed system; and enabling the master node to perform deduplication and fusion operations on the distributed metadata to obtain global lineage data corresponding to the full lifecycle data of various types of data in the heterogeneous data. This application can achieve accurate collection of heterogeneous data, can perform comprehensive metadata collection on the full lifecycle data of various types of heterogeneous data in a distributed system, and can effectively construct lineage relationships among heterogeneous data in a distributed system.
Need to check novelty before this filing date? Find Prior Art