A metadata-driven data bloodline automatic tracking method and system

Abstract

The application provides a metadata-driven data bloodline automatic tracking method and system, and relates to the field of data tracking. Business node data, multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata and timing update cycle metadata are collected; the business node data is encoded based on the multi-level storage path metadata and is filtered based on the standardized format; the business node data is classified in combination with the node semantic similarity metadata, so as to distinguish homologous and homogeneous nodes, homologous and heterogeneous nodes, heterologous and homogeneous nodes, heterologous and heterogeneous nodes and independent business nodes; and the business node is marked in combination with the data update anomaly record metadata and the timing update cycle metadata, so as to distinguish normal update in a period, update anomaly in the period and non-update state across the period, and trace the node; and the mass, dynamic, multi-period and high-reliability data bloodline tracing requirements of an enterprise are met.

Description

Technical Field

[0001] This invention relates to the field of data tracking, and more specifically, to a method and system for automatic data lineage tracking based on metadata-driven approaches. Background Technology

[0002] Enterprise business data exhibits characteristics of multi-source heterogeneity, multi-level storage, high-frequency iteration, and cross-system flow. The entire lifecycle of data, from collection, processing, transformation, output to storage, generates massive business nodes. The flow relationships between nodes, version changes, and abnormal fluctuations directly determine data quality, data reliability, and business traceability. Data lineage tracing, as a core foundational capability of data governance, enables data source querying, flow chain reconstruction, anomaly location, and version tracing. Currently, mainstream data lineage tracing technologies are mainly divided into two categories: one is the hard parsing tracing method based on code parsing and script crawling, which solidifies data flow relationships by parsing ETL scripts and SQL statements; the other is the static lineage construction method based on basic metadata tags, which completes node association based on data storage paths and basic field information. Existing technologies have many inherent defects in practical implementation: First, traditional hard parsing lineage tracing methods rely heavily on fixed ETL scripts and business flow configurations, resulting in extremely poor adaptability to multi-source heterogeneous data sources such as databases, data warehouses, and middleware. After business processes and data fields are iterated, the fixed links directly fail, and the dynamic adaptation capability is weak. Second, existing technologies rely on matching nodes only from a single dimension, which generally leads to problems such as node confusion, mismatch, and disordered lineage links.

[0003] In summary, existing data lineage tracing technologies suffer from weak adaptability to multi-source data and lack standardized node filtering mechanisms, making it difficult to meet enterprises' needs for massive, dynamic, multi-cycle, and highly reliable data lineage tracing. Summary of the Invention

[0004] The purpose of this invention is to provide a metadata-driven automatic data lineage tracking method and system. By combining semantic similarity and format features for joint segmentation and combining abnormal data update cycles to distinguish different tracking modes, this invention solves the problems of weak multi-source data adaptability and lack of standardized node filtering mechanisms in existing data lineage tracking technologies, thereby meeting the needs of enterprises for massive, dynamic, multi-cycle, and highly reliable data lineage tracing.

[0005] The embodiments of the present invention are implemented as follows: This application provides a metadata-driven automatic data lineage tracking method, comprising the following steps: S1, collecting business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for each node; S2, encoding all the business node data based on the multi-level storage path metadata; S3, filtering the business node data based on the standardized format metadata; and classifying the business node data in conjunction with the node semantic similarity metadata to distinguish between nodes of the same origin and similarity, and nodes of the same type. Source heterogeneous nodes, heterogeneous homogeneous nodes, heterogeneous heterogeneous nodes, and independent business nodes; S4, combining the data update anomaly record metadata and the timed update cycle metadata, mark the homogeneous homogeneous nodes, homogeneous homogeneous nodes, heterogeneous heterogeneous nodes, and independent business nodes to distinguish between normal updates within the cycle, abnormal updates within the cycle, and no updates across cycles; S5, for nodes that update normally within the cycle, establish an incremental tracking link; for nodes that update abnormally within the cycle, generate an independent traceability cycle; for nodes that do not update across cycles, complete the continuous traceability time link.

[0006] In step S1, the collection of business node data specifically includes: extracting corresponding operation, status, and result data for data collection, data processing, data output, and data storage business nodes.

[0007] The metadata of the multi-level storage path includes a hierarchical structure of servers, databases, data tables, and fields.

[0008] In step S3, the filtering of the business node data based on the standardized format metadata specifically includes: the standardized format metadata includes field type, storage format, required fields, data precision, and encoding format; invalid business node data is filtered out according to the field type, storage format, required fields, data precision, and encoding format.

[0009] In step S3, classifying the business node data based on the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes specifically includes: using the TF-IDF semantic similarity algorithm to perform coarse classification of the business node data based on the node semantic similarity metadata, aggregating homogeneous business nodes with semantic relevance greater than a preset threshold, filtering out semantically irrelevant heterogeneous business nodes, and independent business nodes; and further classifying the filtered homogeneous and heterogeneous business nodes based on the standardized format metadata to distinguish between homogeneous and heterogeneous business nodes.

[0010] In step S4, the data update anomaly record metadata and the timed update cycle metadata are combined to mark the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes and independent business nodes to distinguish the three types of node states: normal update within the cycle, abnormal update within the cycle and no update across cycles. Specifically, the data update anomaly record metadata includes the time of occurrence of the abnormal data and the type of abnormality. Based on the metadata of the timed update cycle, collect the homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes of each preset update cycle. Based on the time and type of the abnormal data, obtain the status of the abnormal data as normal update within the cycle, abnormal update within the cycle, and no update across cycles.

[0011] A metadata-driven automatic pedigree tracing system includes: Data acquisition module: Collects business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for the corresponding nodes; Data encoding module: Encodes all business node data based on the metadata of the multi-level storage path; Data classification module: Filters the business node data based on the standardized format metadata; classifies the business node data by combining the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes; Status marking module: Combining the metadata of the data update anomaly record and the metadata of the timed update cycle, the same-source homogeneous node, the same-source heterogeneous node, the heterogeneous homogeneous node, the heterogeneous heterogeneous node and the independent business node are marked to distinguish between normal updates within the cycle, abnormal updates within the cycle and no updates across cycles. Data tracking module: For nodes that update normally within the period, establish an incremental tracking link; for nodes that update abnormally within the period, generate an independent tracing period; for nodes that have not been updated across periods, complete the continuous tracing time link.

[0012] An electronic device includes: a memory for storing one or more programs; a processor; and, when the one or more programs are executed by the processor, implements the aforementioned metadata-driven automatic data lineage tracing method.

[0013] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned metadata-driven automatic data lineage tracing method.

[0014] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects: A metadata-driven automatic data lineage tracing method includes the following steps: S1, collecting business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for each node; S2, encoding all the business node data based on the multi-level storage path metadata; S3, filtering the business node data based on the standardized format metadata; and classifying the business node data by combining the node semantic similarity metadata to distinguish between homogeneous nodes and heterogeneous nodes. S4. Based on the metadata of the data update anomaly record and the metadata of the timed update cycle, mark the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes to distinguish between normal updates within the cycle, abnormal updates within the cycle, and no updates across cycles; S5. For nodes that update normally within the cycle, establish an incremental tracking link; for nodes that update abnormally within the cycle, generate an independent traceability cycle; for nodes that do not update across cycles, complete the continuous traceability time link.

[0015] This application implements standardized format invalid node screening, and simultaneously addresses the issues of poor multi-source data adaptation and cross-cycle lineage breakage caused by the lack of a standardized node filtering mechanism through similarity and secondary format subdivision. It uniquely encodes all business nodes based on storage path metadata, achieving unified node identity confirmation; adaptively executes incremental tracking, independent anomaly tracing, and cross-cycle link completion logic according to different node states, completing fully automated data lineage tracking; and links anomaly records with update cycle metadata to achieve precise labeling and layering of three types of node update states, realizing systematic, automated, and high-precision lineage tracking. This invention forms a progressive closed-loop tracking system through pre-format screening, two-dimensional node classification, refined state layering, and differentiated tracing improvements, significantly improving the accuracy, completeness, and automation rate of lineage tracking. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a flowchart of the metadata-driven automatic data lineage tracking method of Embodiment 1 of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other. Example

[0021] Please see Figure 1 , Figure 1 The image shows a metadata-driven automatic data lineage tracking method provided in this application, which includes the following steps: S1, collecting business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for the corresponding nodes; S2, encoding all the business node data based on the multi-level storage path metadata; S3, filtering the business node data based on the standardized format metadata; and classifying the business node data in conjunction with the node semantic similarity metadata to distinguish between nodes of the same origin and similarity. S4. Based on the metadata of the data update anomaly record and the metadata of the timed update cycle, mark the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes to distinguish between normal updates within the cycle, abnormal updates within the cycle, and no updates across cycles; S5. For nodes that update normally within the cycle, establish an incremental tracking link; for nodes that update abnormally within the cycle, generate an independent traceability cycle; for nodes that do not update across cycles, complete the continuous traceability time link.

[0022] Data lineage, also known as data traceability chain, refers to the entire lifecycle of data flow and relationship from original collection, processing, transformation and transmission, indicator calculation to final storage and output. It is used to record the data source, dependencies, change trajectory and scope of impact, and is the core basis for data governance, data quality verification and data anomaly location.

[0023] Incremental Tracing Link: A lightweight lineage link generated for nodes that update normally within a cycle. It only collects incremental data change information within the current update cycle, without loading the full historical data, achieving efficient and real-time short-term data lineage tracking. Independent Tracing Period: For nodes with update anomalies, a separate tracing time interval is generated outside the fixed time period. Starting from the moment the anomaly occurs, it accurately locates the entire process of data flow, operation, and change during the anomaly period, achieving precise source tracing and location of the anomaly. Cross-Cycle Link Completion: For business nodes that have not been updated for a long time, it retrieves historical metadata from multiple cycles and the node's association relationship, repairing the lineage breakage caused by cross-cycle data iteration and node inactivity, and constructing a continuous and complete full lifecycle data lineage link.

[0024] This application implements standardized format invalid node screening, and simultaneously addresses the issues of poor multi-source data adaptation and cross-cycle lineage breaks caused by the lack of a standardized node filtering mechanism through similarity and secondary format subdivision. It uniquely encodes all business nodes based on storage path metadata, achieving unified node identity confirmation; adaptively executes incremental tracking, independent anomaly tracing, and cross-cycle link completion logic according to different node states, completing fully automated data lineage tracking; and links anomaly records with update cycle metadata to achieve precise labeling and layering of three types of node update states, realizing systematic, automated, and high-precision lineage tracking. This invention forms a progressive closed-loop tracking system through pre-format screening, dual-dimensional node classification, refined state layering, and differentiated traceability improvements, significantly improving the accuracy, completeness, and automation rate of lineage tracking.

[0025] In step S1, the collection of business node data specifically includes: extracting corresponding operation, status, and result data for data collection, data processing, data output, and data storage business nodes.

[0026] The metadata of the multi-level storage path includes a hierarchical structure of servers, databases, data tables, and fields.

[0027] In step S3, the filtering of the business node data based on the standardized format metadata specifically includes: the standardized format metadata includes field type, storage format, required fields, data precision, and encoding format; invalid business node data is filtered out according to the field type, storage format, required fields, data precision, and encoding format.

[0028] In step S3, classifying the business node data by combining the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes specifically includes: using the TF-IDF semantic similarity algorithm to perform coarse classification of the business node data based on the node semantic similarity metadata, aggregating homogeneous business nodes with semantic relevance greater than a preset threshold, filtering out heterogeneous business nodes with semantic irrelevant data, and independent business nodes; Based on the standardized metadata format, the filtered homogeneous business nodes and heterogeneous business nodes are further classified to distinguish between homogeneous business nodes and heterogeneous business nodes.

[0029] In step S4, the data update anomaly record metadata and the timed update cycle metadata are combined to mark the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes to distinguish between three types of node states: normal update within the cycle, abnormal update within the cycle, and no update across cycles. Specifically, the data update anomaly record metadata includes the anomaly occurrence time and anomaly type of the abnormal data; based on the timed update cycle metadata, the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes for each preset update cycle are collected; and based on the anomaly occurrence time and anomaly type of the abnormal data, the states of normal update within the cycle, abnormal update within the cycle, and no update across cycles are obtained.

[0030] Step S1 is applied in large enterprise data warehouse governance scenarios, providing real-time access to all business nodes across the entire business system chain, covering the entire process from user behavior collection, business data cleaning, indicator aggregation and output, report data output, to distributed data storage. For each business node, it automatically captures metadata from four layers of storage paths, such as server IP address, data warehouse layered database name, business data table name, and core business field name. Simultaneously, it collects metadata on field types, encoding storage formats, and data precision for each node, extracts semantic similarity metadata from the node's business description text, and synchronously records metadata on abnormal records for each node's preset daily update cycle, historical update errors, data missing, update timeouts, and other anomaly types. This completes the comprehensive collection and aggregation of metadata across all dimensions, providing complete data support for subsequent node filtering, classification, labeling, and tracking.

[0031] Step S2, based on the four-layer, multi-level storage path metadata (server, database, data table, and field) collected in Step S1, uniquely encodes all business nodes using a hierarchical concatenation hash encoding method. Specifically, the server identifier, database number, data table sequence number, and field code are concatenated in a fixed hierarchical order, and a 64-bit unique node code is generated using the SHA-256 algorithm, achieving unique identification and authorization for all business nodes. For example, similar business fields in different data tables under the same server, or business nodes from the same source across servers, can be distinguished by hierarchical encoding to differentiate storage path differences. This completely solves the problems of node duplication, confusion, and mismatch caused by the lack of unified identifiers in traditional nodes, ensuring the uniqueness and accuracy of subsequent lineage associations.

[0032] Step S3 presets invalid node filtering rules based on field type, storage format, required fields, data precision, and encoding format. This removes invalid business nodes with missing fields, incorrect encoding formats, excessive data precision, or empty required fields, while retaining valid business nodes with compliant formats. A preset threshold of TF-IDF semantic similarity can be set to 0.7. Semantic calculations are performed on the business descriptions and functional tag texts of valid nodes. Nodes with a similarity greater than 0.7 are classified as homogeneous business nodes, those less than 0.7 as heterogeneous business nodes, and isolated nodes with no semantic connection are marked as independent business nodes. Based on this, a second subdivision is set using standardized format metadata such as field type, storage format, and data precision: homogeneous business nodes with completely identical format parameters are classified as homogeneous nodes, while those with different format parameters are classified as heterogeneous nodes; heterogeneous business nodes with matching format parameters are classified as heterogeneous nodes, while those with completely different format parameters are classified as heterogeneous nodes. Through format filtering and semantic dual-layer classification, accurate division of five types of nodes is achieved, solving the problems of mismatched nodes and chaotic links caused by traditional single-dimensional matching.

[0033] In step S4, the preset update cycle for each business node is 24 hours, and the node update status is statistically analyzed on a daily basis. For the five categories of categorized business nodes, the metadata of historical anomaly records is aligned one by one: if a node completes data updates within the current update cycle without errors or anomalies, it is marked as a normal update status within the cycle; if there are records of update timeouts, missing data, format errors, or calculation anomalies within the current cycle, and the time of the anomaly falls within the current cycle interval, it is marked as an abnormal update status within the cycle; if a node does not generate any update records or data iterations for two or more consecutive preset update cycles, it is marked as a non-updated status across cycles. Through precise matching of update cycles with anomaly times and anomaly types, a refined hierarchical marking of the update status of all nodes is achieved, providing accurate status basis for differentiated lineage tracing.

[0034] Step S5, for business nodes marked as normally updated within the period, captures only the incremental update data of the current period based on the node's unique code and classification type. It automatically stitches together the upstream and downstream node relationships to generate a lightweight incremental lineage tracing link, eliminating the need to repeatedly load all historical data and significantly improving tracing efficiency. For nodes with abnormal updates within the period, it breaks away from the fixed scheduled update cycle and generates an independent tracing cycle starting from the time of the anomaly. This accurately traces back the entire process of data flow, node operations, and data changes during the anomaly period, pinpointing the source and scope of the anomaly. For nodes that have not been updated across periods, it retrieves complete metadata from multiple historical periods, completes the node relationship links and time nodes of adjacent periods, and repairs the lineage link breakage problem caused by cross-period data iteration in traditional technologies, forming a continuous and complete full lifecycle data lineage tracing link. Example

[0035] The metadata-driven automatic kinship tracing system provided in this application includes: Data acquisition module: Collects business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for the corresponding nodes; Data encoding module: Encodes all business node data based on the metadata of the multi-level storage path; Data classification module: Filters the business node data based on the standardized format metadata; classifies the business node data by combining the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes; Status marking module: Combining the metadata of the data update anomaly record and the metadata of the timed update cycle, the same-source homogeneous node, the same-source heterogeneous node, the heterogeneous homogeneous node, the heterogeneous heterogeneous node and the independent business node are marked to distinguish between normal updates within the cycle, abnormal updates within the cycle and no updates across cycles. Data tracking module: For nodes that update normally within the period, establish an incremental tracking link; for nodes that update abnormally within the period, generate an independent tracing period; for nodes that have not been updated across periods, complete the continuous tracing time link.

[0036] The data encoding module incorporates a hierarchical hash encoding algorithm. It receives multi-level storage path metadata from the data acquisition module and generates a unique, irreversible, exclusive code for each business node according to a fixed hierarchical logic of server-database-datatable-field, storing it in the node identity database. For newly added or iterated business nodes, it automatically identifies storage path changes and updates the node code in real time, avoiding conflicts between old and new node identities and achieving unified management of node identities throughout their entire lifecycle.

[0037] The data classification module has a built-in format filtering rule base and TF-IDF semantic analysis model. First, it removes garbage node data with abnormal format and invalid data through the standardized format metadata database. Then, it completes the coarse classification of nodes by node semantic similarity metadata through the semantic analysis model, dividing them into homogeneous, heterogeneous, and independent nodes. Finally, it completes a secondary fine classification by combining the format metadata, accurately distinguishing five types of business nodes.

[0038] The status marking module is pre-configured with node update cycle rules and anomaly judgment criteria, and it associates real-time update cycle metadata (node ​​historical update records) and anomaly record metadata (anomaly logs). For five different types of nodes, the update status is checked cycle by cycle. Based on the time of anomaly occurrence and the type of anomaly, the node update status is accurately determined, and three status tags are automatically assigned: normal update, update anomaly, and no update across cycles. The status tags are bound to the node codes and stored, realizing dynamic updates and precise stratification of the status of all nodes, and providing a classification basis for differentiated lineage tracing logic.

[0039] The data tracking module incorporates three differentiated lineage tracking logics, automatically adapting to the corresponding tracking mode based on node status tags. For normally updated nodes, it quickly builds incremental lineage links, improving daily data traceability efficiency; for abnormal nodes, it initiates independent traceability cycles to accurately review the entire abnormal process; for nodes that haven't been updated across cycles, it automatically retrieves historical metadata to complete broken links. Ultimately, it generates a complete, continuous, and traceable multi-dimensional data lineage map, supporting users in querying data sources, flow paths, change records, and anomaly locations, achieving fully automated and high-precision data lineage tracking.

[0040] In summary, this application provides a metadata-driven automatic data lineage tracking method and system: S1, collecting business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for each node; S2, encoding all the business node data based on the multi-level storage path metadata; S3, filtering the business node data based on the standardized format metadata; and classifying the business node data in conjunction with the node semantic similarity metadata to distinguish between nodes of the same origin and similarity, and nodes of the same type. Source heterogeneous nodes, heterogeneous homogeneous nodes, heterogeneous heterogeneous nodes, and independent business nodes; S4, combining the data update anomaly record metadata and the timed update cycle metadata, mark the homogeneous homogeneous nodes, homogeneous homogeneous nodes, heterogeneous heterogeneous nodes, and independent business nodes to distinguish between normal updates within the cycle, abnormal updates within the cycle, and no updates across cycles; S5, for nodes that update normally within the cycle, establish an incremental tracking link; for nodes that update abnormally within the cycle, generate an independent traceability cycle; for nodes that do not update across cycles, complete the continuous traceability time link.

[0041] This application implements standardized format invalid node screening, and simultaneously addresses the issues of poor multi-source data adaptation and cross-cycle lineage breakage caused by the lack of a standardized node filtering mechanism through similarity and secondary format subdivision. It uniquely encodes all business nodes based on storage path metadata, achieving unified node identity confirmation; adaptively executes incremental tracking, independent anomaly tracing, and cross-cycle link completion logic according to different node states, completing fully automated data lineage tracking; and links anomaly records with update cycle metadata to achieve precise labeling and layering of three types of node update states, realizing systematic, automated, and high-precision lineage tracking. This invention forms a progressive closed-loop tracking system through pre-format screening, two-dimensional node classification, refined state layering, and differentiated tracing improvements, significantly improving the accuracy, completeness, and automation rate of lineage tracking.

[0042] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A metadata-driven automatic data lineage tracing method, characterized in that, Includes the following steps: S1. Collect business node data, including the multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata of the corresponding nodes. S2. Encode all the business node data based on the metadata of the multi-level storage path; S3. Filter the business node data based on the standardized format metadata; The business node data is classified by combining the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes. S4. Combining the metadata of the data update anomaly record and the metadata of the timed update cycle, mark the homogeneous nodes, the homogeneous nodes, the heterogeneous nodes and the independent business nodes to distinguish between normal updates within the cycle, abnormal updates within the cycle and no updates across cycles. S5. For nodes that update normally within the period, establish an incremental tracking link; for nodes that update abnormally within the period, generate an independent tracing period; for nodes that have not been updated across periods, complete the continuous tracing time link.

2. The metadata-driven automatic data lineage tracking method as described in claim 1, characterized in that, In step S1, the collection of business node data specifically includes: extracting corresponding operation, status, and result data for data collection, data processing, data output, and data storage business nodes.

3. The metadata-driven automatic data lineage tracking method as described in claim 1, characterized in that, The metadata of the multi-level storage path includes a hierarchical structure of servers, databases, data tables, and fields.

4. The metadata-driven automatic data lineage tracking method as described in claim 1, characterized in that, In step S3, the filtering of the business node data based on the standardized format metadata specifically includes: The standardized metadata includes field type, storage format, required fields, data precision, and encoding format; invalid business node data is filtered out based on the field type, storage format, required fields, data precision, and encoding format.

5. The metadata-driven automatic data lineage tracking method as described in claim 1, characterized in that, In step S3, classifying the business node data based on the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes specifically includes: Based on the node semantic similarity metadata, the TF-IDF semantic similarity algorithm is used to coarsely classify the business node data, aggregate the same-origin business nodes with semantic relevance greater than a preset threshold, filter out semantically irrelevant heterogeneous business nodes, and independent business nodes. Based on the standardized metadata format, the filtered homogeneous business nodes and heterogeneous business nodes are further classified to distinguish between homogeneous business nodes and heterogeneous business nodes.

6. The method for automatic data lineage tracking based on metadata-driven methods as described in claim 1, characterized in that, In step S4, the data update anomaly record metadata and the timed update cycle metadata are combined to mark the homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes to distinguish between three types of node states: normal updates within the cycle, abnormal updates within the cycle, and no updates across cycles. Specifically, this includes: The metadata of the data update anomaly record includes the time of occurrence and type of the anomaly data. Based on the metadata of the timed update cycle, collect the homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes of each preset update cycle. Based on the time and type of the abnormal data, obtain the status of the abnormal data as normal update within the cycle, abnormal update within the cycle, and no update across cycles.

7. A metadata-driven automatic lineage tracing system, characterized in that, include: Data acquisition module: Collects business node data, including multi-level storage path metadata, standardized format metadata, node semantic similarity metadata, data update anomaly record metadata, and timed update cycle metadata for the corresponding nodes; Data encoding module: Encodes all business node data based on the metadata of the multi-level storage path; Data classification module: Filters the business node data based on the standardized format metadata; The business node data is classified by combining the node semantic similarity metadata to distinguish between homogeneous nodes, homogeneous nodes, heterogeneous nodes, heterogeneous nodes, and independent business nodes. Status marking module: Combining the metadata of the data update anomaly record and the metadata of the timed update cycle, the same-source homogeneous node, the same-source heterogeneous node, the heterogeneous homogeneous node, the heterogeneous heterogeneous node and the independent business node are marked to distinguish between normal updates within the cycle, abnormal updates within the cycle and no updates across cycles. Data tracking module: For nodes that update normally within the period, establish an incremental tracking link; for nodes that update abnormally within the period, generate an independent tracing period; for nodes that have not been updated across periods, complete the continuous tracing time link.

8. An electronic device, characterized in that, include: Memory, used to store one or more programs; processor; When the processor executes the one or more programs, it implements a metadata-driven automatic data lineage tracing method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a metadata-driven automatic data lineage tracking method as described in any one of claims 1-6.

Images