Data blood relationship automatic tracking system based on large model

Abstract

The invention relates to the technical field of data processing, particularly discloses a data blood relationship automatic tracking system based on a large model, and aims to solve the problems that a traditional method is difficult to adapt to complex business logic, cannot capture deep semantic dependency and is low in cross-system automation degree. The system comprises a data perception and acquisition module, a metadata semantization reconstruction module, a multi-granularity dependency analysis module, a dynamic causal reasoning engine and a blood relationship map self-evolution maintenance module, metadata semantics are analyzed through a large model, a multi-granularity dependency relationship is identified, and the blood relationship map self-evolution maintenance module is established in combination with a causal reasoning verification and map self-updating mechanism. Full-link, high-precision and self-adaptive data blood relationship automatic discovery and maintenance are realized, and the coverage range, the accuracy and the automation level of blood relationship tracking are effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to a data bloodline automatic tracking system based on a large model. BACKGROUND

[0002] In the field of data management and governance, ensuring clear data sources and traceable flow processes is a key link to guarantee data quality and credibility. Data bloodline analysis, as one of the core technologies of data governance, aims to track the complete flow path and transformation process of data from the source to the final consumer terminal. Its importance is increasingly highlighted in the context of data-driven decision-making and increasing compliance requirements.

[0003] The data bloodline automatic tracking system based on a large model is an important development direction of current data governance technology. This technology aims to use the powerful semantic understanding and reasoning capabilities of large models to automatically identify and associate the dependency relationships between data at different processing stages, thereby constructing a complete and accurate data bloodline graph to support data credibility assessment, impact analysis, and problem tracing.

[0004] Existing technologies mainly rely on rule engines or traditional machine learning methods for bloodline analysis, which have significant limitations: rule methods are difficult to cover complex and variable business logic and data transformation scenarios, resulting in broken bloodline chains or insufficient accuracy; traditional machine learning models have weak generalization ability when dealing with large-scale, heterogeneous data sources, and limited ability to capture deep semantic relationships. As the data ecosystem becomes increasingly complex, data flows frequently across systems and platforms, and existing methods are difficult to achieve end-to-end automated bloodline discovery and maintenance, still requiring a lot of manual intervention for rule configuration and result verification. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the defects of the existing data bloodline tracking method, which relies on pre-set rules or traditional machine learning models, and is difficult to adapt to complex and variable business logic, cannot effectively capture deep semantic dependency relationships, and has insufficient automation and accuracy in cross-system data flow scenarios. The purpose of the present application is to provide a data bloodline automatic tracking system based on a large model, which realizes full-link, high-precision, and adaptive bloodline relationship automatic discovery and continuous updating from data sources to consumer terminals by constructing a collaborative technology architecture that integrates multi-modal data perception, semantic dependency graph construction, dynamic causal reasoning, and graph self-evolution maintenance.

[0006] The technical solution of this invention is that the system includes a data perception and acquisition module, a metadata semantic reconstruction module, a multi-granularity dependency parsing module, a dynamic causal reasoning engine, and a lineage graph self-evolution maintenance module. The data perception and acquisition module continuously monitors and acquires metadata streams, data operation logs, data schema change records, and business processing scripts from heterogeneous data sources. This module further integrates a connector management submodule, capable of adapting to various data storage and processing environments, including relational databases, data warehouses, data lakes, stream processing platforms, and file storage systems, and outputs the data to the metadata semantic reconstruction module in a unified time-series data format. The metadata semantic reconstruction module receives the raw metadata stream from the data perception and acquisition module and calls the built-in large language model to perform semantic parsing and structural reconstruction of unstructured or semi-structured metadata entries. This module first performs entity recognition and attribute extraction on the metadata entries, and then uses the contextual understanding capability of the large language model to map the extracted entities and attributes to predefined metadata ontology concepts, generating standardized semantic metadata records with timestamps, data source identifiers, entity types, and confidence scores. The multi-granularity dependency parsing module connects to the metadata semantic reconstruction module to automatically identify dependencies between data entities at different processing stages based on standardized semantic metadata records. This module further includes a syntax-level dependency analysis submodule and a semantic-level dependency inference submodule. The syntax-level dependency analysis submodule extracts table-level, column-level, and row-level data flow relationships by parsing structured query language operations, data transformation scripts, and task scheduling dependencies. The semantic-level dependency inference submodule uses a large language model to perform deep semantic analysis on business logic descriptions, data processing annotations, and field mapping documents to identify implicit dependency links based on business meaning and output dependency triples with dependency type and strength weights. The dynamic causal reasoning engine receives dependency triples from the multi-granularity dependency parsing module and performs causal association verification and spurious dependency filtering based on temporal semantic metadata records. This engine constructs a temporal causal graph model, which identifies and strengthens lineage edges with high causal confidence by calculating the statistical correlation between the order of appearance of data entities and the change propagation path, while eliminating spurious dependencies with inconsistent temporal order or statistical correlation below a preset threshold. The kinship graph self-evolution and maintenance module is connected to the dynamic causal reasoning engine to integrate causally verified kinship relationships and construct a global data kinship graph. This module further implements a real-time update and consistency maintenance mechanism for the graph. By continuously monitoring changes in the data environment, it triggers local recalculation of affected subgraphs and reverification of dependencies to ensure that the kinship graph remains synchronized with the real data state. This module also provides a graph query interface, supporting multi-dimensional kinship path retrieval and visualization by data entity, time range, or dependency type.

[0007] As one embodiment of the present invention, the large language model built into the metadata semantic reconstruction module adopts a pre-trained language model that has been fine-tuned with massive metadata corpus and data governance knowledge. The module performs the following processing flow for each metadata entry: First, it performs text cleaning and segmentation to identify key descriptive fragments in the metadata entry; then, it inputs each descriptive fragment into the large language model, and the model outputs the corresponding entity type, attribute key-value pairs and relational predicates; finally, it performs structured assembly of the model output according to the predefined metadata ontology to generate a standardized semantic metadata record containing data entity identifiers, attribute sets, data operation context and timestamps.

[0008] As one embodiment of the present invention, the workflow of the semantic-level dependency inference submodule includes: obtaining business description text and data processing annotations related to the target data entity from the metadata semantic reconstruction module; calling a large language model to perform semantic encoding and relation extraction on the text, identifying the source data entity, target data entity, and transformation logic mentioned in the description; calculating semantic similarity and analyzing contextual association based on the transformation logic to generate semantic dependency hypotheses; and finally fusing the semantic dependency hypotheses with the output of the syntax-level dependency analysis submodule, and assigning a comprehensive strength weight based on model confidence and contextual support to each dependency relationship.

[0009] As one embodiment of the present invention, the construction process of the temporal causal graph model in the dynamic causal reasoning engine is as follows: data entities are used as nodes, dependencies are used as directed edges, and time window constraints and causal strength coefficients are attached to each edge; the engine calculates the time lag mutual information and Granger causality index between changes in the source entity and changes in the target entity by analyzing historical metadata records, thereby quantifying the causal strength; for any dependency triple, only when its causal strength coefficient exceeds the preset significance threshold and the temporal order conforms to the causal law, is the dependency confirmed as a valid lineage edge and retained in the graph.

[0010] As one embodiment of the present invention, the local recomputation mechanism of the lineage graph self-evolution maintenance module is implemented by an event-driven architecture. The module subscribes to data change events published by the data perception and acquisition module, including table structure changes, data processing task updates, or data source connection status changes. When such events are received, the module automatically identifies the affected nodes and edges in the lineage graph, and schedules the multi-granularity dependency parsing module and the dynamic causal reasoning engine to re-parse and verify the causality of the affected subgraph. Then, the verified results are merged into the global lineage graph in an incremental update manner to ensure the real-time performance and accuracy of the graph.

[0011] As one embodiment of the present invention, the system also includes a strategy management module for configuring the granularity level of kinship tracing, the confidence threshold of causal inference, and the triggering conditions for graph updates; users can customize kinship discovery strategies through this module to adapt to the data governance needs of different business scenarios.

[0012] As one embodiment of the present invention, the system adopts a distributed microservice architecture, with each core module running as an independent service and communicating asynchronously through a high-performance message middleware, thereby ensuring the scalability and processing efficiency of the system in high-concurrency and large-data-volume scenarios.

[0013] Compared with the prior art, the positive effects of the present invention are as follows: First, by leveraging the deep semantic understanding capabilities of the large language model, the system can automatically extract implicit dependencies from unstructured business descriptions and data annotations, significantly expanding the coverage and depth of lineage discovery and effectively addressing the inherent limitations of rule-based methods in dealing with complex business logic.

[0014] Secondly, by introducing a dynamic causal reasoning engine, the system can perform causal verification and eliminate false positives based on the statistical correlation of time-series data changes, which greatly improves the accuracy and reliability of the kinship map and avoids misleading kinship links caused by correlation without causation in traditional methods.

[0015] Furthermore, the system has the ability to self-evolve and maintain the pedigree map, which can respond to real-time changes in the data environment and automatically perform local map updates and consistency checks, realizing continuous automation of end-to-end pedigree tracking and significantly reducing manual maintenance costs.

[0016] In summary, this invention provides a high-precision, fully automated data lineage tracing solution that can adapt to complex and dynamic data ecosystems, providing a solid technical foundation for data governance, data credibility assessment, and compliance auditing. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall technical architecture of the data lineage automatic tracing system based on a large model proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of multi-granularity dependency parsing and dynamic causal reasoning in this invention. Detailed Implementation

[0018] Example 1 Please refer to the attached document. Figure 1The overall technical architecture of this system consists of a data perception and acquisition module, a metadata semantic reconstruction module, a multi-granularity dependency parsing module, a dynamic causal reasoning engine, and a kinship graph self-evolution and maintenance module, all connected sequentially. Each module communicates asynchronously through a high-performance message middleware, ensuring the system's processing efficiency and scalability under high-volume concurrent data scenarios.

[0019] The data awareness and acquisition module, as the system's input, is responsible for continuously monitoring and collecting various metadata information from heterogeneous data sources. This module deploys multiple dedicated connectors that are compatible with mainstream data storage and processing environments such as relational databases, data warehouses, data lakes, stream processing platforms, and file storage systems. Each connector internally implements specific protocol parsing logic. For example, the connector for relational databases captures Data Definition Language (DDL) and Data Manipulation Language (DML) operations by listening to binary logs or system table change records; the connector for data warehouses obtains table structure changes and data processing task information by querying information schema views or parsing load logs; and the connector for stream processing platforms collects real-time data processing topology relationships by subscribing to data stream metadata topics. All connectors are equipped with data buffering and flow control mechanisms, automatically triggering reconnection strategies and data compensation processes when changes in data source connection status are detected. The raw data collected by the connectors includes metadata streams, data operation logs, data schema change records, and business processing scripts. After timestamp alignment and format verification, this data is uniformly converted into a standard time-series data format by the connector management submodule. This time-series data format includes five fixed fields: data source identifier, acquisition time point, operation type code, raw data content block, and data checksum. The data checksum is generated using a cyclic redundancy check algorithm to ensure data transmission integrity. The converted time-series data stream is asynchronously pushed to the input interface of the metadata semantic reconstruction module via a message queue.

[0020] The metadata semantic reconstruction module receives standardized time-series data streams from the data perception and acquisition module and calls the built-in large language model to perform deep semantic parsing on unstructured or semi-structured metadata entries. This module employs a pre-trained language model fine-tuned with massive metadata corpora and data governance knowledge, boasting hundreds of billions of parameters and powerful natural language understanding and structured information extraction capabilities. The module has a three-stage processing pipeline: the first stage is a text preprocessing unit, which performs character encoding normalization, special symbol filtering, text segmentation, and key description fragment recognition on the input metadata entries; the second stage is a semantic parsing unit, which inputs the preprocessed text fragments into the large language model, which encodes the context based on an attention mechanism and outputs parsing results in three dimensions: entity type labels, attribute key-value pairs, and relational predicates; the third stage is a structured assembly unit, which performs semantic mapping and logical verification on the model output according to a predefined metadata ontology concept system, generating standardized semantic metadata records containing globally unique identifiers for data entities, attribute sets, data operation context descriptions, and high-precision timestamps. The records are stored using a unified serialization format. Each record contains twenty standard fields, covering core metadata elements such as data entity identifier, entity type enumeration value, attribute key-value pair mapping table, data operation type, operation timestamp, data source identifier, and confidence score. The confidence score is calculated by combining the probability distribution output by the large language model with the matching degree of the metadata ontology concept, and is used to characterize the reliability of the semantic metadata record. All standardized semantic metadata records are written to a distributed columnar storage database, and dedicated indexes are created for subsequent modules to support efficient queries.

[0021] The multi-granularity dependency resolution module is connected to the output of the metadata semantic reconstruction module via a high-speed data bus. It is responsible for automatically identifying the dependencies between data entities at different processing stages based on standardized semantic metadata records. Please refer to the appendix. Figure 2This module internally consists of two core processing units running in parallel: a syntax-level dependency analysis submodule and a semantic-level dependency inference submodule. The syntax-level dependency analysis submodule employs syntax tree-based parsing technology to perform static analysis and dynamic tracing of structured query language operations, data transformation scripts, and task scheduling configuration files. This submodule first constructs an abstract syntax tree to represent the data operation logic, and then extracts three types of syntax-level dependencies—table-level dependencies, column-level mapping relationships, and row-level filtering conditions—by traversing the syntax tree nodes. Specifically, for structured query language operations, the submodule parses select lists, conditional expressions, and join clauses to determine the field-level correspondence between the source and target tables; for data transformation scripts, the submodule analyzes function call chains and variable passing paths to identify intermediate dependencies in the data processing process; and for task scheduling dependencies, the submodule parses the predecessor-successor relationships in the directed acyclic graph of tasks to determine the execution order of the data processing pipeline. All extracted syntax-level dependencies are converted into a unified triple format, containing three basic elements: source entity identifier, target entity identifier, and dependency type.

[0022] The semantic dependency inference submodule focuses on mining implicit semantic dependencies from business logic description text and data processing annotations. This submodule's workflow comprises four strictly sequentially executed stages: The first stage is the context acquisition stage, which retrieves business requirement documents, data processing annotation text, and field mapping description documents related to the target data entity from the metadata semantic reconstruction module; the second stage is the semantic encoding stage, which calls a large language model to perform deep semantic encoding on the acquired text, generating a high-dimensional semantic vector representation, and simultaneously identifies the source data entity, target data entity, and data transformation logic description mentioned in the text through relation extraction technology; the third stage is the dependency hypothesis generation stage, which constructs a set of semantic dependency hypotheses based on cosine similarity calculation between semantic vectors and context association analysis, with each hypothesis containing a source entity, target entity, transformation logic description, and an initial confidence score; the fourth stage is the dependency fusion stage, which fuses the semantic dependency hypotheses with the triples output by the syntactic dependency analysis submodule, and calculates a comprehensive strength weight for each dependency relationship based on model confidence, context support, and syntactic evidence consistency using a weighted voting algorithm. The final output dependency triple contains five complete attributes: source entity identifier, target entity identifier, dependency type enumeration value, strength weight value, and dependency discovery timestamp.

[0023] The dynamic causal inference engine receives a stream of dependency triples from the multi-granularity dependency parsing module and performs causal association verification and spurious dependency filtering based on temporally sequenced semantic metadata records. The core of this engine is a temporal causal graph model, which uses data entities as nodes and dependencies as directed edges, attaching two key attributes to each edge: a time window constraint and a causal strength coefficient. The construction of the temporal causal graph model involves three core computational steps: The first step is time series alignment, where the engine retrieves historical change records of relevant data entities from the metadata semantic reconstruction module, constructing a timestamped sequence of data entity states. The second step is causal index calculation, which analyzes the time lag relationship between source entity state changes and target entity state changes, calculating two key statistical indicators: time-lag mutual information and the Granger causality index. Time-lag mutual information quantifies the degree of information association between source entity changes and target entity changes at a specific time delay, while the Granger causality index uses a vector autoregression model to test whether the historical state of the source entity has statistical significance in predicting the current state of the target entity. The third step is causal verification decision-making, which calculates the causal strength coefficient based on the above indicators. The specific calculation formula is as follows: Where C represents the causality strength coefficient. Let G represent the time lag mutual information, G represent the Granger causality index, and α and β be weighting coefficients, empirically set to 0.6 and 0.4, respectively. For any dependency triple, a dependency is considered a valid lineage edge and retained in the temporal causal graph model only if its causal strength coefficient exceeds the preset significance threshold of 0.75, and the time order constraint that the source entity's state change time is earlier than the target entity's state change time is satisfied. All eliminated pseudo-dependencies are recorded in the invalid dependency log for system auditing and analysis.

[0024] The lineage graph self-evolution and maintenance module, as the system output, connects to the result output interface of the dynamic causal inference engine. It is responsible for integrating causally verified lineage relationships and constructing a global data lineage graph. This module uses an attribute graph model to represent lineage relationships. Nodes in the graph represent data entities, and edges represent lineage dependencies. Both nodes and edges can carry any number of attribute key-value pairs. Internally, the module implements a complete event-driven architecture to support real-time updates and consistency maintenance of the graph. This architecture includes four core components: an event listener, an impact range analyzer, a subgraph recalculation scheduler, and an incremental merger. The event listener continuously subscribes to the data change event stream published by the data awareness and acquisition module. These events include seven types, such as table structure change events, data processing task update events, and data source connection status change events. When a data change event is received, the impact range analyzer quickly identifies the affected node and edge sets in the lineage graph based on a graph traversal algorithm, forming the boundaries of the affected subgraphs. The subgraph recalculation scheduler then dynamically schedules the multi-granularity dependency resolution module and the dynamic causal inference engine to re-execute dependency resolution and causal verification calculations for the data entities involved in the affected subgraphs, based on the size and complexity of the affected subgraphs. Finally, the incremental merger merges the re-verified lineage relationships into the global lineage graph in a transactional manner, ensuring consistency of query operations during graph updates. This module also provides a complete set of graph query interfaces, supporting multi-dimensional lineage path retrieval by data entity identifier, time range, and dependency type enumeration value. The search results can be used by the visualization engine to generate an interactive lineage graph, clearly showing the complete flow path and transformation relationships of data from the source to the consumer terminal.

[0025] The system also includes a policy management module, which serves as a control plane component and establishes configuration channels with all core modules. The policy management module provides a graphical configuration interface, allowing administrators to dynamically adjust the granularity of lineage tracing, including table-level, column-level, and row-level tracing modes; set confidence thresholds for causal inference, adjustable from 0.5 to 0.95; and define trigger conditions for graph updates, including immediate, delayed, and batch update strategies. All configuration parameters are stored in a versioned manner, supporting configuration rollback and audit trail functionality.

[0026] At the system deployment level, this embodiment adopts a distributed microservice architecture, encapsulating the data awareness and acquisition module, metadata semantic reconstruction module, multi-granularity dependency resolution module, dynamic causal reasoning engine, lineage graph self-evolution maintenance module, and policy management module as independently deployed microservices. Each microservice is deployed on a containerized platform, achieving service discovery and load balancing through a service mesh. Microservices communicate asynchronously through a high-performance message middleware, using binary encoding in the message protocol to improve transmission efficiency. The system supports horizontal scaling, dynamically adjusting the number of instances of each microservice based on the number of data sources and metadata throughput to ensure stable operation under high-concurrency scenarios.

[0027] During system operation, the data perception and acquisition module continuously monitors changes in the data source. When a new data operation is detected, it immediately collects relevant metadata and sends it to the metadata semantic reconstruction module. The metadata semantic reconstruction module uses a large language model to perform semantic parsing on the metadata, generating standardized semantic metadata records. The multi-granularity dependency parsing module simultaneously receives the semantic metadata records and identifies potential dependencies through syntactic analysis and semantic inference. The dynamic causal reasoning engine performs causal verification on the dependencies, filtering out false dependencies. The kinship graph self-evolution and maintenance module finally updates the verified kinship relationships to the global graph and provides kinship query services externally through a query interface. The entire processing flow forms a closed-loop feedback, ensuring that the kinship graph always remains synchronized with the real data state.

[0028] Example 2 This embodiment provides an alternative implementation scheme for an automatic data lineage tracking system based on a large model, focusing on the optimized configuration and adaptation adjustments for specific business scenarios under resource-constrained environments. Please refer to the appendix. Figure 1 With appendix Figure 2 This embodiment retains the core architecture of embodiment 1, and optimizes the implementation strategies of multiple modules accordingly.

[0029] In this embodiment, the data perception and acquisition module employs a lightweight connector design. The connectors only implement the most basic data acquisition functions, omitting advanced monitoring and buffering mechanisms. The connector management submodule introduces an on-demand activation strategy, activating the corresponding connector only when the data source undergoes actual changes, significantly reducing the system's persistent resource consumption. The metadata semantic reconstruction module uses a medium-sized language model with hundreds of billions of parameters, greatly reducing computational resource consumption while maintaining sufficient semantic understanding capabilities. The text preprocessing unit of this module incorporates a compression algorithm, performing lossy compression on metadata entries before inputting them into the language model, further reducing processing overhead.

[0030] In this embodiment, the multi-granularity dependency resolution module adjusts the execution order of the syntax-level dependency analysis submodule and the semantic-level dependency inference submodule. The system first runs the semantic-level dependency inference submodule to generate initial dependency hypotheses, and then only initiates syntax-level dependency analysis on the data entities involved in these hypotheses, avoiding the computational waste caused by performing syntax parsing on all data entities. The semantic-level dependency inference submodule employs knowledge distillation technology to transfer the relation extraction capabilities of large language models to small, specialized models, achieving efficient semantic analysis in offline environments.

[0031] In this embodiment, the dynamic causal inference engine simplifies the construction process of the time-series causal graph model. The engine calculates only the Granger causality index as the sole basis for the causal strength coefficient, omitting the time-lag mutual information calculation step. The causal verification decision threshold is adjusted to 0.7, improving processing speed within an acceptable range of accuracy loss. The engine introduces a sliding window mechanism, retaining only the most recent three months of historical metadata records for causal analysis, reducing data storage and computational pressure.

[0032] In this embodiment, the kinship graph self-evolution maintenance module adopts a lazy update strategy instead of a real-time update mechanism. The module still listens for data change events, but instead of immediately triggering recalculation, it places update tasks in a priority queue and processes them in batches during system idle periods. Recalculation of affected subgraphs uses an approximate algorithm, only re-verifying directly affected kinship edges, without traversing the entire subgraph. The graph query interface supports approximate retrieval of kinship paths, improving response speed through pre-calculation and caching of frequently used query results.

[0033] In this embodiment, the strategy management module includes several preset optimization configuration templates, including high-precision mode, balanced mode, and high-performance mode. Users can switch between different modes with a single click according to their actual business needs, and the system will automatically adjust the parameter configurations of each module. High-precision mode is suitable for scenarios with extremely high requirements for lineage accuracy, such as auditing; balanced mode is suitable for daily data governance work; and high-performance mode is suitable for scenarios with limited resources or where it is necessary to quickly obtain preliminary lineage relationships.

[0034] At the system deployment level, this embodiment supports a hybrid deployment scheme, deploying computationally intensive modules such as the metadata semantic reconstruction module and the dynamic causal inference engine in the cloud, while deploying data-aware modules in the local environment, with data transmission conducted through a secure channel. This deployment approach ensures both processing power and meets the requirements for data privacy and compliance.

[0035] This embodiment is particularly suitable for data governance scenarios in small and medium-sized enterprises, providing reliable data lineage tracing capabilities even with limited infrastructure investment. Through intelligent resource scheduling and algorithm optimization, the system achieves a good balance between accuracy and performance, providing a practical solution for data lineage management in resource-constrained environments.

Claims

1. An automatic kinship tracing system based on a large model, characterized in that, include: The data perception and acquisition module is used to continuously monitor and collect metadata streams, data operation logs, data pattern change records, and business processing scripts from heterogeneous data sources. This module further integrates a connector management submodule, which can adapt to various data storage and processing environments and output to the metadata semantic reconstruction module in a unified time-series data format. The metadata semantic reconstruction module is connected to the data perception and acquisition module and is used to receive the raw metadata stream. This module first performs entity recognition and attribute extraction on the metadata entries, and then uses the context understanding capability of the large language model to map the extracted entities and attributes to the predefined metadata ontology concept, and generates standardized semantic metadata records with timestamps, data source identifiers, entity types and confidence scores. The multi-granularity dependency parsing module, connected to the metadata semantic reconstruction module, is used to automatically identify the dependency relationships between data entities at different processing stages based on standardized semantic metadata records; this module further includes a syntax-level dependency analysis submodule and a semantic-level dependency inference submodule. The syntax-level dependency analysis submodule extracts data flow relationships at the table, column, and row levels by parsing structured query language operations, data transformation scripts, and task scheduling dependencies. The semantic-level dependency inference submodule uses a large language model to perform deep semantic analysis on business logic descriptions, data processing annotations, and field mapping documents, identify implicit dependency links based on business meaning, and output dependency triples with dependency type and strength weights. The dynamic causal reasoning engine, connected to the multi-granularity dependency parsing module, receives dependency triples and performs causal association verification and false dependency filtering based on temporal semantic metadata records. The engine constructs a temporal causal graph model, which identifies and strengthens kinship edges with high causal confidence by calculating the statistical correlation between the order of appearance of data entities and the change propagation path, while eliminating false dependencies with inconsistent temporal order or statistical correlation below a preset threshold. The kinship graph self-evolution and maintenance module is connected to the dynamic causal reasoning engine and is used to integrate causally verified kinship relationships and build a global data kinship graph.

2. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The built-in large language model in the metadata semantic reconstruction module adopts a pre-trained language model that has been fine-tuned with massive metadata corpus and data governance knowledge. The module performs the following processing flow for each metadata entry: First, it performs text cleaning and segmentation to identify key descriptive fragments in the metadata entry; then, it inputs each descriptive fragment into the large language model, and the model outputs the corresponding entity type, attribute key-value pairs and relational predicates; finally, it performs structured assembly of the model output according to the predefined metadata ontology to generate a standardized semantic metadata record containing data entity identifiers, attribute sets, data operation context and timestamps.

3. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The workflow of the semantic-level dependency inference submodule includes: obtaining business description text and data processing annotations related to the target data entity from the metadata semantic reconstruction module; calling the large language model to perform semantic encoding and relation extraction on the text, identifying the source data entity, target data entity, and transformation logic mentioned in the description; calculating semantic similarity and analyzing contextual association based on the transformation logic to generate semantic dependency hypotheses; and finally, fusing the semantic dependency hypotheses with the output of the syntax-level dependency analysis submodule, and assigning a comprehensive strength weight based on model confidence and contextual support to each dependency relationship.

4. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The construction process of the temporal causal graph model in the dynamic causal reasoning engine is as follows: data entities are used as nodes, dependencies are used as directed edges, and time window constraints and causal strength coefficients are attached to each edge; the engine calculates the time lag mutual information and Granger causality index between changes in the source entity and changes in the target entity by analyzing historical metadata records, thereby quantifying the causal strength; for any dependency triple, only when its causal strength coefficient exceeds the preset significance threshold of 0.75 and the temporal order conforms to the causal law, is the dependency confirmed as a valid lineage edge and retained in the graph.

5. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The local recomputation mechanism of the lineage graph self-evolution maintenance module is implemented by an event-driven architecture. This module subscribes to data change events published by the data perception and acquisition module, including table structure changes, data processing task updates, or data source connection status changes. When such events are received, the module automatically identifies the affected nodes and edges in the lineage graph and schedules the multi-granularity dependency parsing module and the dynamic causal reasoning engine to re-parse and verify the causality of the affected subgraph. Then, the verified results are merged into the global lineage graph in an incremental update manner.

6. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The system also includes a strategy management module for configuring the granularity level of kinship tracing, the confidence threshold for causal inference, and the triggering conditions for graph updates. Users can customize kinship discovery strategies through this module to adapt to the data governance needs of different business scenarios.

7. The automatic kinship tracking system based on a large model according to claim 1, characterized in that, The system is deployed using a distributed microservice architecture, with each core module running as an independent service. It communicates asynchronously through a high-performance message middleware, thereby ensuring the system's scalability and processing efficiency in high-concurrency, high-data-volume scenarios.

8. The automatic bloodline tracking system based on a large model according to claim 2, characterized in that, The text cleaning and segmentation process of the metadata semantic reconstruction module includes character encoding normalization, special symbol filtering, text segmentation, and key descriptive fragment recognition operations.

9. The automatic kinship tracking system based on a large model according to claim 3, characterized in that, When the semantic dependency hypothesis generated by the semantic-level dependency inference submodule is fused with the triples output by the syntactic-level dependency analysis submodule, a weighted voting algorithm is used to calculate a comprehensive strength weight for each dependency relationship based on model confidence, context support, and consistency of syntactic evidence.

10. The automatic kinship tracking system based on a large model according to claim 4, characterized in that, The specific formula for calculating the causal strength coefficient is: causal strength coefficient equals 0.6 multiplied by the time lag mutual information plus 0.4 multiplied by the Granger causality index.

Images