A data bloodline full-link automatic tracking and management method and system

Abstract

The application discloses a kind of data blood relationship full-link automatic tracking and management method and system.The present application is unified modeling data entity, field, conversion operator and business rule with knowledge graph, field-level conversion semantics is captured by bypass runtime probe, and is reduced to " field-transform-business rule " triplet to build explicit blood relationship graph;Further, with explicit topology as constraint, according to topological time sequence rank direction, through the confidence propagation with restart mechanism, similar candidate is inferred implicit blood relationship according to entity identity, directional derivation, three-way flow to be verified;Compliance label is propagated by three-state model, and is solidified as governance level atlas, supports SPARQL compliance audit query;Finally, versioning of parameter and ontology is realized through artificial verification feedback, and closed-loop update is realized.The present application significantly improves the accuracy of field-level implicit blood relationship completion, effectively reduces the misconnection rate and direction error rate, realizes compliance state automatic determination and full-link closed-loop management.

Description

Technical Field

[0001] This invention relates to the fields of data governance and knowledge graph technology, specifically to a knowledge graph-based method for automatic field-level data lineage tracing, implicit relationship inference, and compliance governance across the entire data acquisition-processing-consumption chain. It can be applied to scenarios such as government and public data sharing, data element circulation, and data compliance auditing. Background Technology

[0002] Data lineage is used to depict the flow and dependencies of data from its source through processing and transformation to consumption, and is a core foundational capability for data governance and compliance auditing. Currently, mainstream lineage tools can be broadly categorized into three types: The first category is lineage tools based on metadata registration and static code parsing, which extract inter-table or inter-column dependencies by parsing the abstract syntax tree of SQL, ETL (Extract-Transform-Load) configurations or stored procedures; The second type is a lineage collection scheme based on runtime hooks, which attaches listeners when the computing engine is running to collect read and write events at the job level and table level. The third category is a data catalog platform based on a unified metadata model and graph storage, which stores kinship relationships in a graph structure and provides a visual representation.

[0003] In practical applications, data often undergoes multiple levels of field-level semantic transformations from collection to consumption, including aggregation, derivation, filtering, joining, cleaning, and data anonymization, and is subject to various business rules. Existing technologies have the following shortcomings: 1. Coarse lineage granularity: Most tools only record table-level or dataset-level dependencies; some tools that support column-level lineage only record a simple mapping of "which input columns the output columns come from", losing the key semantics of "what transformation operators are used and what business rules are followed", and cannot answer "how the value of a certain field is calculated and what rules are used to generate it".

[0004] 2. Separation of lineage semantics from business rules: The semantics of data transformation are not structured and associated with the underlying business rules such as definitions, desensitization strategies, and access rules. The lineage diagram can only show the flow direction and cannot support compliance traceability.

[0005] 3. Reliance on static parsing, implicit lineage missing: Unable to detect implicit lineage generated by manual cross-system transfer, external import, or third-party components without source code, which easily leads to broken lineage chains.

[0006] 4. Insufficient compliance audit capabilities: It is difficult to perform automated reasoning and querying to determine "where sensitive fields go" and "the upstream root cause path of data problems", and auditing relies on manual investigation.

[0007] 5. Lack of closed-loop governance capabilities: Once pedigrees are collected, they are stored statically. Manual verification results and new processing modes cannot be fed back to the collection and inference stages, and the pedigree map gradually becomes distorted as the process evolves.

[0008] The more critical issue is that in complex processing environments involving multiple operations, cross-source data, and data backfilling, existing technologies are prone to misjudging entities with the same relationship (same name, alias, mirror field) as derived relationships, or generating reverse lineage edges due to insufficient evidence. It is difficult to balance the accuracy of field-level implicit lineage completion with the reliability of automatic governance. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a knowledge graph-driven method for automatic tracking and governance of the entire data lineage chain. This method addresses the problems of existing data lineage tools, such as coarse granularity, separation of semantics and business rules, missing implicit lineage, weak compliance audit capabilities, and lack of closed-loop governance. In particular, it solves the technical problems of low accuracy in field-level implicit lineage completion, easy confusion between entity identity and derivation relationships, error-prone reverse lineage, and poor reliability of automatic governance in multi-operation cross-source environments.

[0010] According to a first aspect of the present invention, a knowledge graph-driven method for automatic tracking and governance of the entire data lineage is provided, comprising the following steps: S1. Define the ontology class and association relationship oriented towards data lineage, construct the lineage ontology, and define the ontology axiom set; instantiate the externally input data asset metadata into data entity nodes and field nodes, structure the externally input business rule documents into business rule nodes, complete the node instance initialization, and output the initial node set, operator type library and business rule node set. S2. Based on the operator type library and business rule node set, the externally input ETL / ELT job is parsed into a directed acyclic graph and a topology number is assigned to each operation node. The operator type of each operation node is matched according to the operator type library. Lineage probes are injected into the input and output sides of each operator in a bypass observer manner. At the same time, the scheduling batch number and job dependency level of the job are parsed to generate a probe registry and a directed acyclic graph of the job structure. S3. During job execution, the lineage probe is used to collect field-level conversion semantics and generate a conversion semantic event stream; S4. Based on the transformation semantic event stream and the lineage ontology, each transformation event is reduced to a field-transformation-business rule triple and written into the knowledge graph to generate an explicit lineage graph; S5. Based on all field nodes and their attributes and structural context in the explicit lineage graph, calculate multidimensional semantic similarity for field pairs that are cross-job, cross-source, and have no explicit lineage direct connection, retain field pairs whose comprehensive similarity reaches a preset similarity threshold, and generate a set of similarity candidate edges. S6. First, pre-calculate the temporary direction for each similar candidate edge according to the topological temporal rank, and then perform confidence iterative propagation with a restart mechanism using the explicit lineage graph as the propagation skeleton. Finally, process all similar candidate edges according to three mutually exclusive branches: entity sameness, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges, and pending verification candidates. S7. Based on the ontology axioms set, perform consistency verification and logical completion on the merged graph, and propagate compliance tags along the lineage link according to the three-state model of original sensitivity level, inherited risk level, and desensitized shareable level, to generate a governance-level lineage knowledge graph and consistency report. S8. In response to an external audit request, based on the governance-level lineage knowledge graph, perform sensitive data flow audit, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and output the audit results. S9. Generate structured governance strategy instructions based on the audit results and issue them to external execution components. At the same time, update the graph parameters and ontology rules in a versioned manner based on manual verification feedback to form a rollback-capable full-link closed-loop governance.

[0011] According to a second aspect of the present invention, a knowledge graph-driven automatic data lineage tracking and governance system is provided, comprising: The ontology modeling module is used to define ontology classes and relationships oriented towards data lineage, construct lineage ontology, and define ontology axioms; it instantiates externally input data asset metadata into data entity nodes and field nodes, structures externally input business rule documents into business rule nodes, completes node instance initialization, and outputs the initial node set, operator type library, and business rule node set; The probe injection module is used to parse the externally input ETL / ELT job into a directed acyclic graph and assign a topology number to each operation node according to the operator type library and the business rule node set. It matches the operator type of each operation node according to the operator type library and injects lineage probes on the input and output sides of each operator in a bypass observer manner. At the same time, it parses the scheduling batch number and job dependency level of the job and generates a probe registry and a directed acyclic graph of the job structure. The semantic capture module is used to collect field-level transformation semantics through the lineage probe during job execution and generate a transformation semantic event stream; The explicit lineage construction module is used to standardize each transformation event into a field-transformation-business rule triple based on the transformation semantic event stream and the lineage ontology, and write it into the knowledge graph to generate an explicit lineage graph. The similar candidate generation module is used to calculate multidimensional semantic similarity for field pairs that are cross-job, cross-source and have no explicit direct connection based on all field nodes and their attributes and structural context in the explicit lineage graph, retain field pairs whose comprehensive similarity reaches a preset similarity threshold, and generate a set of similar candidate edges. The implicit lineage inference module is used to first pre-calculate the temporary direction for each of the similar candidate edges according to the topological temporal rank, and then perform confidence iterative propagation with a restart mechanism using the explicit lineage graph as the propagation skeleton; finally, all similar candidate edges are processed according to three mutually exclusive branches: entity sameness, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges and pending verification candidates; The governance-level graph solidification module is used to perform consistency verification and logical completion on the merged graph based on the ontology axiom set, and to propagate compliance tags along the lineage link according to the original sensitive level, inherited risk level, and desensitized shareable level three-state model to generate a governance-level lineage knowledge graph and consistency report. The compliance audit analysis module is used to respond to external audit requests, and based on the governance-level lineage knowledge graph, to perform sensitive data flow audits, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and output audit results. The closed-loop governance feedback module is used to generate structured governance strategy instructions based on the audit results and issue them to external execution components. At the same time, it updates the graph parameters and ontology rules in a versioned manner based on manual verification feedback, forming a rollback-capable full-link closed-loop governance.

[0012] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: 1. Confidence propagation constrained by explicit lineage topology improves implicit lineage recall while significantly reducing false connection rate, and isolated highly similar candidates are effectively suppressed by topological constraints. 2. The topological temporal rank uniquely determines the derivation direction, significantly reducing the reverse rate of directed edges and ensuring the correctness of the direction of root cause tracing and influence surface analysis; 3. The three-tiered mechanism of entity identity, directed derivation, and pending verification reduces the probability of misdirection or misconnection of fields with the same name / alias / mirror, and reduces the workload of manual review; 4. Three-state sensitive level state transition clearly distinguishes between legitimate desensitization and non-compliant downgrade, reduces compliance false alarm rate, and minimizes interference with normal business operations; 5. Bypass probes collect data non-intrusively, minimizing impact on business operations; confidence propagation with restart feature converges quickly, templated queries have low response latency, and are available for engineering applications. 6. The inferred edges, parameters, and governance feedback are versionable and rollbackable, and the lineage graph remains accurate as the link evolves, with historical traceability.

[0013] 7. In terms of application, it can transform compliance judgment from manual investigation to automatic state transition reasoning on the graph, support the refined governance of data classification and grading, and provide traceable technical basis for the record-keeping and regulatory audit of cross-departmental data flow.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0015] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the invention.

[0016] Figure 1 This is a schematic diagram of the overall process of the method of the present invention, showing the input-output relationship and closed-loop feedback path of steps S1 to S9; Figure 2 A schematic diagram of the principle of bloodline probe bypass posting and ternary array is shown, corresponding to steps S2 to S4; Figure 3 This is a schematic diagram of the directional confidence propagation and three-way flow inference principle of topological constraints, corresponding to steps S5 to S6; Figure 4 This is a schematic diagram of the closed loop of three-state sensitive level propagation and audit execution, corresponding to steps S7 to S9. Detailed Implementation

[0017] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms "first," "second," "third," etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, features defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0021] The core concept of this invention is: to unify the modeling of data entities, fields, transformation operators, business rules, and their relationships using a knowledge graph; to capture field-level transformation semantics at runtime using a bypass probe during the ETL / ELT (Extract-Transform-Load / Extract-Load-Transform) process, and to write these semantics into a "field-transformation-business rule" triplet into the knowledge graph, forming a directional explicit lineage topology; and then, using this explicit lineage topology as a skeleton constraint, to pre-compute the topological temporal rank of cross-source candidate fields without explicit direct edges. Temporary directions are propagated with confidence levels under topological constraints. All candidates are then processed into three mutually exclusive branches: entity identity, directed derivation, and pending verification (the first two branches require a confidence level threshold as a necessary condition; those not met are classified as pending verification). The directed derivation branch solidifies the verified temporary directions into formal directed lineage edges. After ontology verification and propagation of compliance labels according to the three-state model of "original sensitive level - inherited risk level - desensitized shareable level," the temporary directions are solidified. Finally, SPARQL queries support compliance auditing and impact surface analysis, outputting policy instructions for execution by external components. Manual verification feedback drives the versionable and rollbackable closed-loop updates of parameters and the ontology.

[0022] like Figure 1 As shown in the figure, the knowledge graph-driven automatic tracking and governance method for the entire data lineage provided by this invention specifically includes the following steps: Step S1: Ontology Modeling Define ontology classes and relationships oriented towards data lineage, construct a lineage ontology containing five ontology classes: data entity classes, field classes, transformation operator classes, business rule classes, and processing operation classes, and define an ontology axiom set; instantiate externally input data asset metadata into data entity nodes and field nodes, structure externally input business rule documents into business rule nodes, complete node instance initialization, and output the initial node set, operator type library, and business rule node set.

[0023] Specifically, in one embodiment, the data asset metadata comes from external data sources, including a list of database tables, field names, data types, field comments, and existing hierarchical classification labels; the business rule documents come from external input, including definitions, desensitization strategies, and access and sharing rules; the ontology axiom set is a pre-defined set of logical constraint rules, including the transitivity of field derivation relationships, the monotonicity of sensitivity levels along the lineage, and the type compatibility constraints of operator inputs and outputs; the operator type library is a pre-defined set of transformation operator subclasses, including six types of operators: aggregation, derivation, filtering, joining, cleaning, and desensitization; and the business rule node set is a structured collection of business rule instances, with each rule containing a rule identifier, rule type, scope of application, rule parameters, priority, and source document information.

[0024] Step S2: Probe injection like Figure 2 As shown, taking the operator type library and business rule node set output in step S1 as input, the externally input ETL / ELT job is parsed into a directed acyclic graph (DAG) and a topology number is assigned to each operation node. The operator type of each operation node is matched according to the operator type library, and lineage probes are injected into the input and output sides of each operator in the manner of bypass observers. At the same time, the scheduling batch number and job dependency level of the job are parsed to generate a probe registry and a directed acyclic graph of the job structure.

[0025] Specifically, the directed acyclic graph is a directed acyclic logical graph with operation nodes as vertices and data flow direction as edges; the topology number is the sequential number assigned to each operation node according to the topology sorting of the directed acyclic graph, with the source end number being the smallest and increasing along the data flow direction, used for step S6 to determine the lineage direction.

[0026] In one embodiment, for SQL / Spark jobs, parsing callbacks are registered in operator nodes such as Project / Aggregate / Filter / Join by rewriting their logical plan, directly extracting output expressions, aggregation keys, join keys, filter predicates, and input / output schemas from the plan node; for visual ETL, listeners are registered through their operator plugin interfaces. A registration record is generated for each probe, registering the probe ID, the job_id to which it belongs, the DAG position and topology sequence number, the operator type, resolvable input / output field references, and associative business rule references (named / labeled according to rules and matched with the business rule node set). Simultaneously, two cross-job ordering information items are parsed and registered from the scheduling system (such as the workflow scheduler's job dependency DAG) for step six to construct the global lexicographical order: the job's scheduling batch number seq_batch and the job's dependency level lvl_job in the job dependency graph.

[0027] Optionally, in step S2, the lineage probe adopts a bypass observer working mode, which only collects field references, runtime modes, transformation expressions and parameter class metadata, without reading actual business data values ​​or modifying data content and business logic; For scenarios where there are no source code components or manually imported classes cannot parse and transform expressions, the probe records the input and output field patterns and source identifiers, marks the operator type as unknown, and leaves it for subsequent inference and completion.

[0028] Step S3: Semantic Capture like Figure 2 As shown, during job execution, the lineage probe injected in step S2 collects field-level transformation semantics and generates a transformation semantic event stream.

[0029] Specifically, in this step, the collection boundaries are first clarified, that is, the probe only collects metadata (field references, runtime schema, transformation expressions and parameters) by default, and does not collect plaintext data values; value distribution statistics are only generated on the operator side in a way that is calculated on-site and desensitized and aggregated when needed (such as histogram binning and cardinality estimation of field values), and only traditional measurement is returned instead of the original value to avoid leakage of specific information.

[0030] When an instrumentation job is scheduled for execution, each probe, based on its registered location, job_id, topology sequence number, and operator type in the probe registry, collects the actual input / output field set, runtime schema, transformation expressions and parameters (aggregate functions and grouping keys, derived expressions, filtering predicates, join keys, desensitization algorithms and strengths, etc.), and local statistics for that operator. It then parses out the field-level mapping of "which input fields, through which operators and expressions, produce which output fields," and references and annotates related business rules according to the rules in the probe registry (those not parsed are set to empty or left for attribution). Each record is appended with an execution timestamp (ts) and a run identifier (run_id).

[0031] In step S3, for scenarios where the conversion expression cannot be parsed at runtime, such as those without source code components or manual import, the lineage probe records the schema and source identifier of its input and output fields, marks the operator type op_type as Unknown, sets the conversion expression expr to empty, and generates an event to be supplemented; after being marked in step S4, the lineage relationship is inferred through the similarity in steps S5 and S6, or it is entered through the manual supplementation interface.

[0032] The output of this step includes: a transformed semantic event stream, where each event is a structured tuple with the following fields in order: job ID (job_id), topology sequence number (ord), scheduling batch number (seq_batch), job dependency level (lvl_job), input field set (F_in), operator type (op_type), transformation expression (expr), operator parameters (params), output field (F_out), associated business rule reference (rule_ref, which can be empty), local value distribution statistics (stat), timestamp (ts), and run ID (run_id); for source fields (without upstream operators), the probe registers their data arrival time / batch time (t_arrive).

[0033] Therefore, the arrival time, scheduling batch number, job dependency level, topology number within the job, and event time required for the topology timing order in step S6 are all explicitly carried by the probe registry and the event stream.

[0034] Step S4: Explicit lineage establishment like Figure 2 As shown, taking the transformation semantic event stream of step S3 and the lineage ontology of step S1 as input, each transformation event is reduced to a field transformation business rule triple and written into the knowledge graph to generate an explicit lineage graph.

[0035] Specifically, in this step, events are read one by one, and for each output field and each of its source input fields, a "field-transformation-business rule" triple is generated. That is, the following four edges are established in the graph: the output field is derived from the source field; the operator instance node corresponding to the event consumes the input field, produces the output field, and is governed by the associated business rule.

[0036] The type of the operator instance node is taken from the operator type of the event, and it records the transformation expression and parameters. When the rule reference of the event is empty, the givenBy edge is not created and the operator node is marked "to be attributed". A Job instance is created or reused according to the job identifier of the event and a "job contains operator" edge is created to solidify the job-operator attribution, providing a basis for the output of the affected job set in step eight.

[0037] Before writing to the knowledge graph, the same "input field - operator type - output field" relationship generated from multiple runs should be merged into one edge, and its occurrence count should be accumulated and recorded as the support occurrence count, which is used as evidence along with the time window; then, the legality is verified according to the ontology relationship definition before being written to the graph database.

[0038] Two types of output are distinguished: those with both rule references and expressions are complete triples; those with empty rule references or operators of type Unknown only generate "field-transformation-to-be-attributed" semi-triples, marked as to-be-attributed / to-be-added, and do not produce dangling outputs. Before writing, deduplication and merging are performed on the same relation that has been run multiple times, and the cumulative occurrence is used as support evidence.

[0039] Step S5: Generating Similar Candidates like Figure 3 As shown, taking all field nodes and their attributes and structural context in the explicit lineage graph of step S4 as input, multidimensional semantic similarity is calculated for field pairs that are cross-job, cross-source and have no explicit lineage direct connection. Field pairs with a comprehensive similarity that reaches a preset similarity threshold are retained to generate a set of similarity candidate edges.

[0040] Specifically, the explicit lineage graph only contains explicit lineages that can be parsed by code / configuration; implicit relationships such as those manually transferred across systems or imported from external sources are still missing. To discover implicit relationships, this step calculates multidimensional semantic similarity as candidate evidence for field pairs that are cross-job or cross-source and currently have no explicit edges directly connecting them (field pairs within the same entity or with existing explicit derivation relationships are skipped, and first filtered by name / type blocks to control scale). The overall similarity is a weighted sum of four components: name similarity, type compatibility, value distribution similarity, and structural context similarity. The weights of each component are non-negative and sum to 1. The values ​​of the four components are determined as follows: 1. Name Similarity: After normalizing the field names and comments by lowercase conversion, delimiter removal, Chinese word segmentation, and synonym merging, the larger value between the normalized edit distance similarity and the word vector cosine similarity is taken, with a value range from 0 to 1; 2. Type compatibility: The value is 1 when the data types of the two fields are completely consistent, a preset constant when lossless conversion is possible, and 0 when the types are incompatible; the lossless conversion means that the data type conversion will not lose information, such as converting an integer to a long integer. 3. Value distribution similarity: Based on the value distribution histograms of the two fields, calculate 1 minus the Jensen-Shannon divergence of the two distributions; the Jensen-Shannon divergence is a numerical index that measures the degree of difference between two probability distributions, and the smaller the difference, the higher the value distribution similarity. 4. Structural Context Similarity: Take the Jaccard coefficient of the set of adjacent fields of each of the two fields along the association relationship (e.g., consumption / production / derivation); the Jaccard coefficient is the ratio of the intersection size to the union size of the two sets, used to measure the degree of overlap between the sets.

[0041] Step S6: Implicit Bloodline Deduction like Figure 3As shown, taking the similar candidate edge set from step S5 and the explicit lineage graph from step S4 as input, a temporary direction is first pre-calculated for each similar candidate edge according to the topological temporal rank. Then, the confidence iterative propagation with a restart mechanism is performed using the explicit lineage graph as the propagation skeleton. Finally, all similar candidate edges are processed according to three mutually exclusive branches: entity sameness, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges, and pending verification candidates.

[0042] Specifically, relying solely on pairwise similarity can easily lead to incorrect connections. This step uses explicit lineage topology as a constraint and similarity as the initial seed to iteratively propagate the confidence level on the graph. A confidence level is assigned to each candidate edge, with its initial value being the seed value obtained by normalizing the candidate's comprehensive similarity.

[0043] Since the candidates are undirected similar pairs, a temporary direction is calculated for each candidate edge before propagation according to the topological temporal rank rule below (the field with the larger rank points to the field with the smaller rank); the temporary direction for edges whose ranks are not comparable is set to "undetermined". This temporary direction is only used for propagation calculation and is not written into the graph; it is solidified into a formal derived edge after the confidence reaches the threshold and falls into the directed derived branch below.

[0044] Propagation occurs between candidate edges. Specifically, if two candidate edges are adjacent or share upstream and downstream connections on the explicit lineage graph, they mutually reinforce each other's confidence; isolated, highly similar candidates are suppressed as they are not reinforced. The reinforcement strength between two candidate edges is determined by a weighted sum of the following factors: the number of upstream fields they share, the number of downstream fields they share, the attenuation value corresponding to the shortest explicit path length (the greater the distance, the smaller the contribution), whether their temporary directions are consistent (consistency adds points, indeterminate directions add no points), and the cumulative support of the explicit edges supporting them; the reinforcement strength is zero when two candidate edges are neither adjacent nor share upstream and downstream connections on the explicit graph. The reinforcement strength of each candidate edge to its related candidate edges is normalized to obtain the propagation weight.

[0045] Iteration Rules: In each round, the new confidence score of each candidate edge is taken as the weighted average of "the confidence scores of its related candidate edges weighted by propagation weights" and "its own initial seed value". The weight of the two is controlled by a restart coefficient between 0 and 1 (the larger the restart coefficient, the more it is biased towards the initial similarity; the smaller the coefficient, the more it is biased towards topology propagation). The confidence score is calculated starting from the initial seed value, and the iteration continues until the change between two adjacent rounds is less than the convergence threshold. Convergence and Anomaly Handling: The above iteration is a contraction process with restart, which has a unique steady state and will inevitably converge. The engineering implementation sets a maximum number of iterations and a convergence threshold. If numerical anomalies occur or convergence is not achieved after exceeding the maximum number of iterations, the process reverts to using only the initial similarity seed and records an alarm.

[0046] For all candidate field pairs in the similar candidate edge set, the following three mutually exclusive branches are used to model the entity identity relation and the directed derivation relation separately, and to clarify the source of the directed implicit edge. The entity identity branch and the directed derivation branch both require the candidate confidence to reach the confidence threshold as a necessary condition. Candidates that do not meet the corresponding branch conditions will enter the verification branch, so that each candidate has a unique exit and those with insufficient confidence also have a legitimate destination.

[0047] The rule for determining three mutually exclusive branches is as follows: 1. Same branch of entity: If the candidate confidence reaches the preset confidence threshold, meets the basic conditions of synonym identification with consistent type, highly overlapping value distribution, and belonging to different namespaces, and hits at least one strong piece of evidence without triggering any rejection conditions, an undirected same relation (sameAs) edge or a directed alias relation (aliasOf) edge is established, and a derived relation (derivesFrom) edge is not established. The strong evidence includes: ① Consistent controlled semantic tags — Both fields are attached to the same semantic tag in the same controlled thesaurus / data element standard (such as the referenced "citizen identity number" standard data element); ② Consistent Sample Fingerprint Matching – The values ​​of the two fields are irreversibly consistent (e.g., salted hashing / format normalization hashing) to obtain a set of sample fingerprints. The matching rate exceeds the preset fingerprint matching threshold (it is recommended to set it to 0.8 to 0.95, only comparing fingerprints and not returning plaintext). ③ Registration mapping evidence – There are registered exchange interfaces or field mapping tables between the source systems that clearly show the correspondence between the two.

[0048] The rejection conditions (if any one of them is met, no alignment edge will be built) include: mutually exclusive semantic tags (such as one being "medical card number" and the other being "citizen identity number"); conflicting rules for value range / dimension / number of digits; the consistent fingerprint matching rate is lower than the above fingerprint matching threshold; the two fields have been distinguished into different roles by explicit blood ties within the same entity.

[0049] 2. Directed Derivation Branch: If the conditions for the same branch of entities are not met, but the temporary direction can be determined, the confidence level reaches the threshold, the type compatibility check is passed, and the anti-loop verification is passed, a directed derivation relationship edge is established according to the temporary direction, marked as an inference edge, and the confidence level, similarity component and verification evidence are retained. 3. Branches to be verified: Candidates that do not meet the above two branch conditions are marked as pending verification and output to the manual verification queue. No formal bloodline edge is established, and the failure conditions and corresponding evidence are recorded.

[0050] The rejection condition for entity identity is only used to prevent the establishment of sameAs or aliasOf relationships. When the rejection condition does not exclude the existence of a derivation relationship between fields (e.g., two fields have different semantic labels and do not constitute the same entity, but one is indeed processed by the other through an encoding / mapping operator), the candidate field pairs still enter the directed derivation branch 2 for judgment. Thus, the determination of entity identity and derivation relationships no longer obscure each other: the entity identity branch absorbs candidates with the same name / alias / mirror field to avoid them being misdirected, and the candidates not absorbed by this branch then determine whether to solidify the directed derivation edge according to the temporary direction and three checks. The two are still mutually exclusive exits.

[0051] The rules for determining the topological temporal rank are as follows: Lineage direction is determined by the topological temporal rank of a field, addressing the problem of determining direction when "field similarity is undirected but derivation relationships are directed": fields with a later rank (later temporal sequence, downstream) are derived from fields with a earlier rank (earlier temporal sequence, upstream). To ensure that the rank can be calculated for both source fields and cross-job fields, the topological temporal rank is uniformly defined as a sorting key that can be compared item by item, and the comparisons are made in the following priority order: (1) The source field (an external source table field that is only used as input and never as output of any operator) is assigned the lowest rank and is compared only with its data arrival time / batch time, so it is always on the source side; (2) The processing field (the field that is first generated as the output of an operator) is compared with four items in sequence: the scheduling batch number of the job to which it belongs, the dependency level of the job in the job dependency graph, the topology number of the field in the DAG within the job (assigned in step S2), and the event time; if the previous item is the same, the next item is compared; (3) When two fields belong to a non-common scheduling spectrum (no common scheduling batch and job dependency graph is unreachable and cannot be compared), they are considered to be incomparable in rank. No directed derived edges are built. Only the same entity relationship is allowed to be established or the field is downgraded to a candidate to be verified. (4) For fields with the same name generated by incremental batch, repeated execution, and backfill tasks, the sorting key at the time of their first generation shall be used as the standard (backfill tasks shall be marked separately and compared with the key of the target batch to be backfilled) to avoid rank drift caused by reruns.

[0052] In many-to-many mapping, aggregation, and splitting scenarios, each "input field - output field" combination of the same operator is directed separately; aliases are not repeatedly inferred after being merged in the same branch of the entity.

[0053] Step S7: Treatment-level spectrum solidification like Figure 4 As shown, taking the merged graph from step S6 and the ontology axiom set from step S1 as input, the merged graph is subjected to consistency verification and logical completion based on the ontology axiom set. After the original sensitivity level is inherited and the risk level is desensitized, the three-state model of the shareable level is propagated along the lineage link to generate a governance-level lineage knowledge graph and a consistency report.

[0054] Specifically, consistency verification and logical completion include: 1. Field-level transitive closure completion: Calculates transitive closures only for derivation relationships between fields, forming reachable relationships that can be directly queried; 2. Conflict Detection and Resolution: Detect conflicts involving cycle dependencies and type compatibility violations, and resolve them according to the principle of prioritizing explicit edges over implicit edges and higher confidence edges over lower confidence edges; 3. Matching of operators to be attributed: Match scores according to applicable scope, and associate the operators to be attributed with the business rules with the highest scores that exceed the threshold; 4. Entity alignment and merging: The same relationship is merged into the same logical field according to the equivalent closure, and all original physical field nodes are retained to support cross-system traceability.

[0055] Regarding the closure constraints of entity identity relations, it should be noted that the equivalence closure calculation of `sameAs` is limited to cross-namespace merging within the same controlled semantic label / data element standard. That is, only when two fields are attached to the same semantic label in the same controlled vocabulary and no veto condition is met, will their `sameAs` relation participate in the equivalent closure. Fields across different controlled standards, even if their names are similar, will not be included in the same equivalence class to prevent unconstrained propagation of closures between heterogeneous namespaces. `aliasOf` represents "alias pointing to a canonical field," and its transitivity only holds within the same logical data domain. For example, if `a` is an alias of `b`, `b` is an alias of `c`, and all three are in the same domain, then `a` is an alias of `c`. Cross-system `aliasOf` is not automatically transitive; cross-system correspondences must be explicitly established through strong evidence such as "registration mapping verification" to avoid erroneous merging of canonical fields due to cross-system concatenation of mirrored fields.

[0056] The three-state model is a compliance judgment model that classifies field sensitivity levels into three categories and performs state transitions along the lineage chain. Its state transition and propagation rules are as follows: 1. Original sensitivity level: The inherent initial sensitivity level of the source field; 2. Inheritance Risk Level: For non-de-identification operators, the inheritance risk level of the output field is the highest level among all the inheritance risk levels of its source fields, and it monotonically does not decrease along the lineage. 3. Shareable level after desensitization: Output fields processed by the approved desensitization operator have their sensitivity level legally downgraded to the preset shareable level; The approved de-identification operator library is a controlled list registered in step S1 and supplementable in step S9. Each approved de-identification operator records its operator identifier and version, applicable input data type, output data type, shareable level after degradation as determined by rules, and inapplicable conditions (such as not recognizing the degradation effect for inputs that do not meet the minimum length or have been truncated). Only when the operator and its parameters meet the strength requirements of the corresponding de-identification rule and the input meets the applicable conditions is it recognized as an "approved de-identification operator" and triggers a legitimate degradation; otherwise, it is treated as a non-de-identification operator. Violation judgment: When the inherited risk level of a field is lower than the highest risk level of its source, and its generating operator is not an approved de-identification operator, it is marked as a violation degradation point; degradation by an approved de-identification operator is marked as a legitimate de-identification point. This semantically distinguishes "legitimate de-identification" from "violation degradation," eliminating the conflict between the two.

[0057] For the candidates to be verified output in step S6, this step only performs checks on the integrity of record fields, traceability of evidence, and consistency of candidate status, and persists them with the graph as candidate records or status nodes; they are not used as formal lineage edges to participate in field-level transmission closure, sensitive-level propagation, or automatic blocking by governance strategies, so that the candidates to be verified can be queried and manually adjudicated without polluting the formal governance graph. The graph is solidified after the verification is passed and the graph is completed.

[0058] This step outputs a governance-level lineage knowledge graph and a consistency report. The governance-level lineage knowledge graph persistently includes complete triples, stateful semi-triples, explicit lineage edges, inferred lineage edges, entity alignment edges, tri-state sensitive labels, and candidate records or state nodes to be verified. Each candidate record to be verified includes associated field pairs, candidate confidence, similarity components, failure conditions, temporary directions or undetermined reasons, and corresponding evidence. Candidate records to be verified do not participate in the transitive closure and sensitivity propagation of formal `derivesFrom` edges, but can be queried in step eight and used as objects for manual confirmation, denial, revocation, and versioning updates in step S9. The consistency report includes resolved conflicts, entity alignment results, remaining attribution / supplementation points, and a list of violation / downgrade points. Both are used by steps S8 and S9.

[0059] Step S8: Compliance Audit Analysis like Figure 4 As shown, taking the governance-level lineage knowledge graph and external audit request from step S7 as input, the system performs sensitive data flow audit, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and outputs an audit result list, impact report, and root cause path.

[0060] Specifically, SPARQL is a standard graph query language for resource-oriented description frameworks, used to perform pattern matching and path traversal on knowledge graphs.

[0061] In this step, based on the provided SPARQL interface and pre-built audit / analysis templates, external audit requests are compiled into graph pattern matching and path traversal on the governance-level graph. The traversal direction is defined as follows: In this invention, the direction of the `derivesFrom` edge is "output field → source field" (i.e., from downstream to upstream). Therefore, traversing along the pointing direction (forward) of the edge reaches the upstream source, and traversing along the reverse pointing direction (reverse / incoming edge) reaches the downstream derivative. Accordingly, the types of audits are as follows: (1) Sensitive data flow audit refers to the process of performing a transitive closure traversal along the reverse edge (incoming edge direction) of derivedsFrom for a given sensitive field, listing all downstream consumption fields, passing operators and the business rules they are subject to, and identifying the path to unauthorized domains or cross-border flows. (2) Root cause tracing refers to traversing along the positive edges (outgoing edges) of derivedFrom, given a problem field, locating the upstream source field and the transformation operators and rules along the path, and outputting the root cause path; (3) Impact analysis refers to the process of traversing the downstream fields and operations affected by the proposed change along the reverse edge and traversing the upstream fields that the field depends on along the forward edge. (4) Governance gap exposure refers to querying needAttribution / Unknown operators, inferred edges with low confidence, and violation downgrade points as a list for manual verification. The query results are labeled in layers according to the nature of the lineage: factual lineage (explicit edges), inferred lineage (inferred edges with confidence), and candidates to be verified (low confidence or to be supplemented), for subsequent actions to distinguish and handle them.

[0062] This step outputs an audit results list (including flow path, hit strategy, and lineage stratification) and an impact report (affected upstream and downstream fields and job sets). Step S9: Closed-loop governance feedback like Figure 4 As shown, a structured governance strategy instruction is generated based on the audit results of step S8 and sent to the external execution component. At the same time, the graph parameters and ontology rules are updated in a versioned manner based on manual verification feedback, forming a rollback-capable full-link closed-loop governance.

[0063] Specifically, this step outputs policy instructions instead of directly modifying data, and delivers them to an external execution component via a standard interface, thus providing a feasible technical path for "blocking unauthorized / cross-border flows": (1) Strategy generation and execution docking - Generate structured strategy instructions based on audit results (action type: block / de-identify / downgrade / alarm / work order; target: field, job, open interface; trigger condition and reference to the rule), and send them to the data gateway, task scheduler or access control system through the strategy executor interface (such as sending field-level access denial rules to the data gateway, sending job pause instructions to the scheduler); the execution component writes back the execution receipt (success / failure / partial success), and if it fails, it is handled according to the preset downgrade strategy (such as downgrading to alarm only and setting a manual work order), and the receipt is written back to the graph as an audit trace.

[0064] (2) Versionable and rollbackable closed-loop update - using manual confirmation / denial as a supervision signal: the implicit edges of confirmation / denial are used as positive / negative samples, and the cross-entropy of edge-level binary classification is used as the loss. The parameter set is fine-tuned online with a bounded step size. The online fine-tuning parameter set includes: weights of four similarity components, similarity threshold, confidence threshold, restart coefficient and directional confirmation coefficient. Each parameter update has upper and lower bounds, sample window, version number and rollback snapshot. Each parameter is updated in small steps with a small learning rate and is only updated within the sliding sample window. The history of the inferred edges that have been written is not silently rewritten - each inferred edge has a version number, status (candidate / confirmed / denied / revoked), evidence and effective time. When manually denied, its status is set to revoked and its weight is reduced instead of being physically deleted, so as to retain audit traceability. The parameter update also records the version. In case of anomaly, it can be rolled back to the previous version parameter set and the corresponding graph snapshot.

[0065] (3) Rewrite the artificial attribution conclusions back into operator-rule associations, and add the new operator patterns to the operator type library and probe recognition rules.

[0066] This step ultimately outputs policy instructions and execution receipts (including failure and degradation records); versioned parameter set increments (injection steps S5 and S6); ontology and probe rule increments (injection steps S1 and S2); and inference edge updates with version and state. Thus, the lineage graph can continuously adapt and its history is traceable and rollbackable during the evolution of the link, forming a closed-loop chain of "modeling—injection—capture—construction—similarity—inference—verification—auditing—feedback".

[0067] The technical solutions of the embodiments of the present invention can first bring about a set of measurable technical effects: (1) The implicit lineage across systems is automatically completed by confidence propagation oriented by the topological temporal rank, with explicit lineage topology as constraint. The confidence is strengthened only when both ends of the candidate are in the lineage reachable / upstream and downstream consistent position, thereby improving the field-level implicit lineage recall rate while reducing the implicit edge false connection rate. Isolated highly similar candidates are suppressed by topological constraints and are no longer trivially connected.

[0068] (2) The derivation direction is uniquely determined by the topological temporal rank, which reduces the reverse rate of directed edges, so that the source tracing and influence surface analysis can be carried out in the correct direction, thereby improving the root cause path integrity rate.

[0069] (3) The three mutually exclusive branches of entity same, directed derivation and pending verification are split to reduce the rate of misdirection or misconnection of the same name / alias / mirror field (such as the trap pair of medical card number and unique identity), and reduce the workload of manual review.

[0070] (4) The three-state sensitivity level along the lineage state transfer distinguishes between “legitimate desensitization” and “illegal downgrade”, reduces the rate of legitimate desensitized fields being falsely reported as illegal downgrade, and reduces interference with normal business operations.

[0071] (5) The probe is inserted in the manner of a bypass observer, only collecting metadata and on-site desensitized statistics. It does not intrude on existing operations and the execution results remain unchanged. Under bypass probe, the impact on business execution is small. The confidence propagation with restart has a limited number of convergence rounds, and the templated audit query response is fast. It has inference and query latency that can be used in engineering.

[0072] (6) The inferred edges, parameters and governance feedback can be versioned and rolled back. The lineage graph remains accurate and historically traceable as the link evolves and is not silently rewritten.

[0073] Based on the above-mentioned technical effects, the present invention further brings several application-level effects: it transforms compliance determination from manual investigation to state transition reasoning that can be automatically calculated on the graph, which facilitates data hierarchical classification and governance, and provides traceable evidence for the record-keeping and regulatory audit of cross-departmental data flow.

[0074] To enable readers to better understand the innovative aspects of this invention, the following details the distinguishing technical features and their effects compared to the closest prior art.

[0075] Distinguishing Feature 1: The basic unit of lineage is a triple of "field—transformation—business rule," rather than the existing binary mapping of "output column originates from input column." Operator instances (recording operator type, transformation expression, and operator parameters) are explicitly modeled as nodes and connected by consumes / produces / derivesFrom / governedBy. Technical Effect: Lineage can answer "based on which transformation and which rule is it obtained," supporting the processing purpose and traceability of compliance audits.

[0076] Distinguishing Feature Two: Implicit lineage is inferred by propagating directional confidence based on the explicit lineage topology obtained from runtime probes, constrained by the topological temporal rank. Unlike similarity flooding and other methods that propagate along the similarity graph itself with no fixed direction, the propagation framework of this invention is constructed from the explicit lineage edges through which the actual data flows, and the direction is uniquely determined by the topological temporal rank. Technical Effects: Confidence is only strengthened when both ends of a candidate are in lineage-reachable / upstream / downstream consistent positions. Isolated highly similar candidates are suppressed, and reverse misjudgments are eliminated. While maintaining recall, false connections are significantly suppressed—this mechanism and its improved accuracy are not readily apparent from general similarity matching.

[0077] Distinguishing Feature 3: For all similar candidates, the system is judged based on three mutually exclusive branches: entity identity (sameAs / aliasOf), directed lineage derivation (derivesFrom), and pending verification (the first two branches require a confidence threshold as a necessary condition; those with insufficient confidence or failing verification are classified as pending verification). "Entity identity" and "lineage derivation" are modeled separately, and their mutual exclusion relationships are clearly defined—at most one of the same candidate is hit. The entity identity branch eliminates candidates with the same name / alias / mirror field that are misdirected. The remaining candidates are then determined based on temporary directions and verification to solidify the directed derivation edge. Technical effect: This avoids mistaking aliases of the same field as derivations, or derivations as the same, improving inference accuracy.

[0078] Distinguishing Feature 4: Three-state sensitivity levels (original / inherited risk / shareable after desensitization) are used for state transfer along lineage, distinguishing between legitimate desensitization and non-compliant downgrades. Technical Effect: Compliance determination is transformed into automatically computed state transfer reasoning on a graph, resolving the semantic conflict of "downgrade equals violation".

[0079] Distinguishing Feature 5: Policy instructions are delivered to external executors via a standard interface, and the inferred edges and parameters form a closed loop that is versionable and rollbackable. Technical Effects: This enables "automatic blocking" to have a feasible technical path, prevents silent rewriting of historical audits, and allows for rollback of anomalies.

[0080] Features two, three, and four above work together: the explicit topology captured by the probe is both the source of the triplet and serves as the propagation framework and orientation basis; the aligned directed lineage participates in the propagation of the three-state label; the audit and feedback are then reinjected into the parameters and ontology through a rollback mechanism, forming a non-obvious overall technical solution.

[0081] In summary, this invention does not use the existing modules side by side, but rather uses the same runtime field-level event as the common input, and generates subsequent data structures and technical effects step by step according to the forced data dependency order of "temporary orientation and topology propagation - three-way flow - formal orientation solidification - governance propagation". Each mechanism is used for different error dimensions, is a prerequisite for each other and cannot be substituted for one another.

[0082] The following example illustrates a specific implementation method using a government / public data sharing scenario.

[0083] A city's big data platform needs to process source data from departments such as social security into datasets for external sharing, and meet the requirements of data classification and compliance auditing, namely, personal information must not be shared beyond authorized scope, and cross-departmental flow must be traced.

[0084] Example 1: End-to-end tracking with implicit lineage and entity alignment A city's big data platform includes a population database `res_population` (fields: `id_card` unique identifier, `name` name, `birth` date of birth) and a social security database `res_social` (fields: `sfz` insured ID card, `salary` contribution base). Assignment J2 de-identifies `id_card` and derives it as `id_mask`, then derives `age` from `birth` and filters for minors, generating an intermediate table `mid_person`. Assignment J3 associates `mid_person` with `res_social` to generate a shared table `share_person`. `res_social.sfz` is manually imported from another system; it shares the same entity identifier as `id_card` but has no coded association, representing an implicit relationship.

[0085] 1. Step S1: Define the ontology and axioms, instantiate the database table field nodes, and structure "unique identity is personal specific information and can only be shared after desensitization" into business rule nodes. Output the initial node set, operator type library and business rule node set. 2. Step S2 parses J2 and J3 as directed acyclic graphs and assigns topological indices. It maps the desensitization function to the Mask operator, the age calculation to the Derive operator, and the association to the Join operator and inserts instruments to generate a probe registry. 3. During step S3, capture conversion events such as the Mask operator input `id_card` and output `id_mask`, and associate them with the "ID card desensitization" rule; the age calculation operator has not yet been associated with any rules. 4. In step S4, reduce the triplet and write it into the explicit lineage graph, and mark the age operator as to be attributed; 5. Step S5 calculates the multidimensional similarity between `id_card` and `sfz`: the semantic labels are both "ID card", the types are the same, the value distributions are highly overlapping, the contexts are similar, and the overall similarity exceeds the threshold, so they are included in the similarity candidate edge set; 6. After the confidence propagation in step S6, if the confidence of both entities reaches the target, they are determined to be the same branch. A `sameAs` (same relation) edge is established, but no derived edge is established. 7. Step S7 completes the field-level transitive closure, matches the "minor protection" rule for the age operator, and propagates the three-state sensitivity level along the lineage: the `id_mask` approved Mask operator is downgraded to the shareable level; 8. Step S8 performs an audit of "id_card flow" to obtain its flow into the shared table after being anonymized, and then links it to the social security domain through the same entity association; 9. Step S9 generates a blocking policy for the unauthorized flow and sends it to the data gateway. After the administrator confirms the entity alignment, it is used as a positive sample fine-tuning parameter set. The version number and confirmed status of this edge are recorded.

[0086] Example 2: Misconnection Inhibition Control The health database fields `card_no` (medical card number) and `id_card` have similar names, types, and digits, making them easily mistakenly linked as the same or derived from each other based solely on pairwise similarity. In step S6, `card_no` and `id_card` are neither reachable from each other in the explicit lineage graph nor do they share upstream or downstream operators, thus failing to obtain topological verification. Their confidence level is suppressed below the threshold, and they are ultimately assigned to the branch awaiting verification for manual judgment, effectively avoiding false connections.

[0087] Example 3: Structured Implementation of Business Rule Nodes and Policy Instructions In the automatic matching of attribution operators in step S7 and the implementation of governance strategies in step S9, business rules and policy instructions are stored and transmitted in a structured format, providing a data foundation for automatic reasoning, matching calculation, and interface integration. Exemplary implementations are described below: Business rule nodes are structured and stored in the knowledge graph according to the ontology definition of this invention. Their fields provide the calculation basis for rule matching scores, conflict resolution, and de-identification effectiveness determination in step S7. In one embodiment, business rule nodes can be stored in JSON format. Taking the "ID card de-identification" rule in this embodiment as an example, its exemplary structure is as follows: { "rule_id":"R-IDCARD-MASK-001", "rule_type":"Desensitization", "scope":{ "table_pattern":["res_*","mid_*"], "field_tag":["citizen identification number"], "ops":["Mask"], "effective":["2024-01-01","2099-12-31"] }, "strength":{"method":"sha256+salt","keep_prefix":0,"keep_suffix":4}, "prio":90, "source_doc": "Article 4.2 of the 'Specifications for De-identification of Government Data Sharing'" } In the above structure, `rule_id` is a globally unique identifier for business rules; `rule_type` is used to distinguish rule categories, including types such as de-identification, caliber, access control, and sharing; the `scope` field defines the applicable boundaries of the rule, including table name matching mode, field semantic tags, applicable operator types, and effective time intervals, which is the core calculation basis for the matching score of the attribution operator in step S7; the `strength` field defines the execution parameters and strength of the rule, and for de-identification rules, it also serves as the standard for determining "approval of de-identification operators" in step S7; `prio` is the rule priority, used to resolve conflicts when the matching scores of multiple rules are close; `source_doc` is used to record the compliance source documents of the rule, supporting end-to-end audit traceability.

[0088] In step S9, the system automatically generates structured governance policy instructions based on the audit results and distributes them to external execution components such as the data gateway and task scheduler via standard interfaces. Taking the unauthorized flow blocking in the above-mentioned ID card de-identification scenario as an example, its exemplary policy instruction structure is as follows: { "policy_id":"P-20240515-0007", "action":"block", "target":{"type":"field","ref":"share_person.id_card","interface":"OpenAPI / gw-share"}, "rule_ref":"R-IDCARD-MASK-001", "condition":"caller domain" Authorized domains and fields without approved de-identification operators", "effective":["2024-05-15T00:00:00","2024-06-15T00:00:00"], "dispatch":{"endpoint":"data-gateway / acl","method":"PUT","async":true}, "on_success":{"receipt":"ACK-2xx","writeback":"Audit record"}, "on_failure":{"code":"5xx / timeout","fallback":"downgraded to alarm-only + generate manual work order","receipt":"NACK"} } In the above structure, `policy_id` is the unique identifier of the policy instruction; `action` defines the type of action to be performed in the governance process, supporting five types of governance actions: blocking, desensitization, downgrading, alarming, and work order; the `target` field defines the target object and interface of the policy; `rule_ref` is associated with the corresponding business rule to ensure that the governance action is traceable and compliant; the `condition` field defines the judgment condition for triggering the policy; the `effective` field defines the effective time window of the policy; the `dispatch` field defines the endpoint, invocation method, and transmission mode of the instruction; `on_success` and `on_failure` define the processing logic after successful and failed execution, respectively: when the execution is successful, the execution receipt is written back to the governance-level graph to form an audit trail; when the execution fails, it is automatically downgraded to an alarm notification and a manual processing work order is generated to avoid the governance action from becoming ineffective.

[0089] The structure of the above strategy instructions is an exemplary implementation and can be adapted and adjusted according to the interface specifications of external execution components without affecting the implementation of the core governance logic of this invention.

[0090] Example 4: End-to-end Orientation Across Operations, Backfilling, and Rank Incomparable Job J4 is a backfill task. It will be run again the next day to backfill the historical `birth` and recalculate `age`. Job J5 in the health domain generates `visit.card_no` independently and has no common scheduling batch with the social security domain.

[0091] Step S6: Calculate the topological time rank: The direction from `birth` to `age` follows the rule of "first generated sort key as the standard, backfilling based on comparison of the target batch being backfilled". The direction will not drift due to reruns, and `age` will still be determined to be derived from `birth`. `card_no` and `id_card` belong to different scheduling families, their ranks are incomparable, their temporary directions are undetermined, no formal derived edges are established, and the "semantic label mutual exclusion" rejection condition is met. They also do not establish entity identity relations, and are ultimately downgraded to candidates to be verified.

[0092] Example 5: Implicit lineage completion for source code-less black-box components The external system manually imports the field `ext_person_id`, whose semantic label is "external personnel identifier," which does not meet the entity identity condition with `id_card`. Job J6 is a third-party component without source code; it processes the shared table field `share_person.person_ref`. The probe can only record the input and output schema and cannot parse the internal expressions. Step S4 only writes semi-triples and does not establish explicit derived edges; Step S5 calculates the similarity between the two: semantically similar names, compatible types, highly overlapping value distributions, and similar contexts. If the overall similarity meets the criteria, it enters the candidate set; Step S6 determines that the two are rank-comparable, have clear temporary directions, meet the confidence criteria, and pass the verification. If so, it is determined to be a directed derived branch, and an inferred derived edge is established and evidence is preserved.

[0093] Example 6: Status Transition of Candidates to be Verified The cross-source field pairs `ext_addr` (external import address) and `addr_norm` (normalized address) meet the overall similarity standard, but they belong to no common scheduling lineage, their ranks are incomparable, and their temporary directions are undetermined.

[0094] Step S6 assigns it to the branch to be verified, generates a record to be verified and outputs it to the manual queue; Step S7 only performs integrity verification, persists it as a state node, and does not participate in closure and sensitivity level propagation; Step S8 governance gap query can list the candidate.

[0095] Step S9, manual feedback, falls into two categories: 1. Administrator confirmation: Solidify as a formal derived edge, mark as manually confirmed, record the basis and version, and use it as a parameter for fine-tuning positive samples; 2. Administrator denial: Marked as revoked, not included in lineage calculation, used as a negative sample reinjection parameter, and the record is permanently retained and not deleted.

[0096] Example 7: Error Suppression of Black-Box Components The black-box component J7 has similar names for its input field `ext_code` and output field `biz_flag`, but their value distributions have low overlap. Step S5 calculates a low overall similarity; Step S6 further verifies that temporary directions are incomparable, or loop prevention checks fail, and the component is ultimately classified as a branch awaiting verification with reasons for failure, submitted for manual review, thus avoiding the creation of incorrect lineage edges.

[0097] Example 8: Technical Effect Verification Test To verify the technical effectiveness of this invention in automatic lineage tracking, implicit relationship inference, and compliance governance at the field level, multiple sets of comparative experiments were conducted using a government data anonymization test dataset. The verification process and results are as follows.

[0098] 1. Experimental Dataset and Operating Environment This embodiment uses a government data anonymization test dataset, which includes 48 database tables, 1260 fields, 85 processing jobs, and 320 operator instances. A field-level lineage standard truth set was formed through manual annotation, containing a total of 2150 real lineage edges, of which 1290 are explicit lineage edges that can be obtained through code logic parsing, and 860 are implicit lineage edges that are manually imported from across systems or generated by components without source code. At the same time, 120 sets of trap field pairs that are prone to misjudgment (such as non-same-origin fields with similar or similar names to medical card numbers and unique identifiers) and 95 sets of entity same-name field pairs were annotated.

[0099] The operating environment uses a 3-node computing cluster, with each node configured with a 16-core CPU and 64GB of memory. Spark 3.6 is used as the execution engine, and probe instrumentation is implemented in its logical plan layer. The knowledge graph is stored using an attribute graph.

[0100] 2. Test parameter settings The core parameters used in this embodiment are as follows: the weights of each component of multidimensional semantic similarity are: name similarity 0.4, type compatibility 0.15, value distribution similarity 0.25, and structural context similarity 0.2; the similarity threshold is 0.6; the confidence propagation restart coefficient is 0.15; the confidence threshold is 0.7; the maximum number of iterations is 100; the convergence accuracy is one in a million; and the strong evidence fingerprint matching threshold is 0.8.

[0101] The above parameters are exemplary values ​​for this government service scenario. In actual applications, they can be adjusted according to the data scale, business scenario, and accuracy requirements.

[0102] 3. Baseline Design To demonstrate the advancement of this invention over existing technologies, five typical existing solutions were selected as baselines for comparison: B1: The traditional static resolution column lineage scheme generates explicit column-level mappings only through code parsing, without performing implicit inference; B2: Column-level mapping + pure pairwise similarity completion scheme, which only supplements implicit edges based on field pairwise similarity, without topological constraints and direction determination; B3: Similarity graph label propagation scheme, which uses label propagation on the similarity graph to infer lineage, with no fixed direction constraints; B4: A lineage inference scheme with topological propagation, which propagates the confidence of topological constraints but does not distinguish between entity identity and derivation relationships; B5: Complete kinship scheme, with complete explicit and implicit kinship inference, but only uses a single sensitive label, without distinguishing between legal desensitization and illegal downgrading.

[0103] 4. Definition of Evaluation Indicators This embodiment sets evaluation indicators from two dimensions: positive effect and negative error. (1) Field-level lineage recall: the number of lineage edges in the solution output that are consistent with the standard truth value / the total number of lineage edges in the standard truth value set, which measures the coverage of lineage relationships; (2) Accuracy: The number of bloodline edges in the solution that match the true value / the total number of bloodline edges in the solution, which measures the accuracy of the inference; (3) Root cause path integrity rate: The number of paths that can be traced back to all real sources in the output field to be traced / the total number of real root cause paths, which measures the end-to-end tracing capability; (4) Implicit edge error rate: The number of newly inferred implicit edges that do not match the true value / the total number of newly inferred implicit edges, which measures the degree of misjudgment in implicit inference; (5) Directed edge reversal rate: the number of directed edges whose direction determination is opposite to the true value / the total number of directed edges in the scheme output, which measures the accuracy of the lineage direction determination; (6) Trap Pair False Positive Rate: The number of trap field pairs that were incorrectly established as blood relations / the total number of trap field pairs, which measures the ability of an entity to distinguish between the same relationship and the blood-derived relationship.

[0104] 5. Experimental Results and Analysis The test results of this embodiment are shown in Tables 1 and 2.

[0105] Table 1 Comparison of Positive Effect Indicators Table 2 Comparison of Reverse Error Indicators The experimental results show that: The field-level lineage recall rate of this invention reaches 93.4%, which is 33.4 percentage points higher than the traditional static parsing scheme. It effectively solves the problem of implicit lineage loss caused by manual cross-system transfer and lack of source code components. At the same time, the precision rate reaches 95.7%, and the implicit edge false connection rate is only 4.3%, which is 17.2 percentage points lower than the false connection rate of the pure similarity completion scheme. This proves that the mechanism of propagating confidence based on explicit topological constraints can effectively suppress false connections while improving the recall rate.

[0106] In terms of directional accuracy, the directed edge reversal rate of this invention is only 1.2%, which is 14.5 percentage points lower than the label propagation scheme without directional constraints. This verifies the effectiveness of the topological temporal rank orientation mechanism and ensures the directional correctness of root cause tracing and impact surface analysis.

[0107] In terms of entity differentiation capability, the trap pair of the present invention has a false positive rate of 5.0%, which is 51.7 percentage points lower than the pure similarity scheme. This shows that the three-stream mechanism can effectively distinguish between entity sameness and bloodline derivation relationship, and significantly reduce the false positive probability of same name and mirror field.

[0108] Further testing was conducted on the performance under different explicit kinship ratios: when the explicit edge ratio was 40%, the recall rate of this invention was 88.7% and the precision rate was 93.6%; when the explicit edge ratio was 80%, the recall rate was 96.2% and the precision rate was 97.1%. It can be seen that the richer the explicit topology, the more significant the precision gain of this invention compared to the pure similarity scheme.

[0109] In terms of compliance determination, the three-state sensitivity level model of this invention can control the rate of false alarms of legitimate desensitized points as violations and downgrades to 2.1%, which is significantly lower than the 29.4% of the single sensitivity label scheme, verifying the role of three-state transition in improving the accuracy of compliance determination.

[0110] In terms of performance, the confidence propagation in this embodiment converges in about 18 rounds, and the total time for implicit lineage inference is about 6.8 seconds; the average response time of the pre-set templated audit query is about 120 milliseconds, which has the real-time capability for engineering implementation.

[0111] 6. Ablation Verification of Core Technology Features To verify the independent technical function of each core technical feature of this invention, ablation tests were performed on the complete scheme, and the core mechanisms were removed one by one before retesting. The results are as follows: (1) The confidence propagation mechanism of removing topological constraints is degraded to a pure similarity threshold scheme: the accuracy rate drops from 95.7% to 78.5%, and the false connection rate increases from 4.3% to 21.5%, proving that topological constraint propagation is the core mechanism for suppressing false connections; (2) Remove the topological temporal rank orientation mechanism and degenerate into label propagation in any direction: the directed edge reverse rate increases from 1.2% to 15.7%, and the root cause path integrity rate decreases from 91.3% to 77.5%, proving that temporal rank orientation plays a decisive role in the correctness of direction; (3) Remove the same branch judgment of entities and mix the judgment of alignment and derivation relationship: the misjudgment rate of trap pairs increased from 5.0% to 38.3%, and the accuracy decreased to 86.1%, proving that the three-way flow mechanism can effectively reduce the misdirection probability of fields with the same name / alias; (4) Remove the three-state sensitivity level distinction and adopt a single sensitivity label: the compliance false alarm rate increased from 2.1% to 29.4%, proving that the three-state model can effectively distinguish between legitimate desensitization and illegal downgrading.

[0112] The above ablation results show that the four core technical features of this invention—topology propagation, temporal rank orientation, three-way flow determination, and three-state compliance propagation—act on different error dimensions, each with its own independent technical effect, and are mutually dependent and irreplaceable, together constituting the complete technical solution of this invention.

[0113] The test results in this embodiment are exemplary results for this dataset. The specific values ​​vary depending on the size of the dataset, the proportion of explicit lineage, and the business scenario, and do not limit the scope of protection of this invention.

[0114] Corresponding to the above method embodiments, an exemplary embodiment of the present invention also provides a knowledge graph-driven automatic tracking and governance system for the entire data lineage chain, comprising: The ontology modeling module is used to define ontology classes and relationships oriented towards data lineage, construct lineage ontology, and define ontology axioms; it instantiates externally input data asset metadata into data entity nodes and field nodes, structures externally input business rule documents into business rule nodes, completes node instance initialization, and outputs the initial node set, operator type library, and business rule node set; The probe injection module is used to parse the externally input ETL / ELT job into a directed acyclic graph and assign a topology number to each operation node according to the operator type library and the business rule node set. It matches the operator type of each operation node according to the operator type library and injects lineage probes on the input and output sides of each operator in a bypass observer manner. At the same time, it parses the scheduling batch number and job dependency level of the job and generates a probe registry and a directed acyclic graph of the job structure. The semantic capture module is used to collect field-level transformation semantics through the lineage probe during job execution and generate a transformation semantic event stream; The explicit lineage construction module is used to standardize each transformation event into a field-transformation-business rule triple based on the transformation semantic event stream and the lineage ontology, and write it into the knowledge graph to generate an explicit lineage graph. The similar candidate generation module is used to calculate multidimensional semantic similarity for field pairs that are cross-job, cross-source and have no explicit lineage direct connection based on all field nodes and their attributes and structural context in the explicit lineage graph, retain field pairs whose comprehensive similarity reaches a preset similarity threshold, and generate a set of similar candidate edges. The implicit lineage inference module is used to first pre-calculate the temporary direction for each of the similar candidate edges according to the topological temporal rank, and then perform confidence iterative propagation with a restart mechanism using the explicit lineage graph as the propagation skeleton; finally, all similar candidate edges are processed according to three mutually exclusive branches: entity same, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges and pending verification candidates; The governance-level graph solidification module is used to perform consistency verification and logical completion on the merged graph based on the ontology axiom set, and to propagate compliance tags along the lineage link according to the original sensitive level, inherited risk level, and desensitized shareable level three-state model to generate a governance-level lineage knowledge graph and consistency report. The compliance audit analysis module is used to respond to external audit requests, and based on the governance-level lineage knowledge graph, to perform sensitive data flow audits, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and output audit results. The closed-loop governance feedback module is used to generate structured governance strategy instructions based on the audit results and issue them to external execution components. At the same time, it updates the graph parameters and ontology rules in a versioned manner based on manual verification feedback, forming a rollback-capable full-link closed-loop governance.

[0115] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for automatic tracking and management of data lineage across the entire chain, characterized in that, Includes the following steps: S1. Define the ontology class and association relationship oriented towards data lineage, construct the lineage ontology, and define the ontology axiom set; instantiate the externally input data asset metadata into data entity nodes and field nodes, structure the externally input business rule documents into business rule nodes, complete the node instance initialization, and output the initial node set, operator type library and business rule node set. S2. Based on the operator type library and business rule node set, the externally input ETL / ELT job is parsed into a directed acyclic graph and a topology number is assigned to each operation node. The operator type of each operation node is matched according to the operator type library. Lineage probes are injected into the input and output sides of each operator in a bypass observer manner. At the same time, the scheduling batch number and job dependency level of the job are parsed to generate a probe registry and a directed acyclic graph of the job structure. S3. During job execution, the lineage probe is used to collect field-level conversion semantics and generate a conversion semantic event stream; S4. Based on the transformation semantic event stream and the lineage ontology, each transformation event is reduced to a field-transformation-business rule triple and written into the knowledge graph to generate an explicit lineage graph; S5. Based on all field nodes and their attributes and structural context in the explicit lineage graph, calculate multidimensional semantic similarity for field pairs that are cross-job, cross-source, and have no explicit lineage direct connection, retain field pairs whose comprehensive similarity reaches a preset similarity threshold, and generate a set of similarity candidate edges. S6. First, pre-calculate the temporary direction for each similar candidate edge according to the topological temporal rank, and then perform confidence iterative propagation with a restart mechanism using the explicit lineage graph as the propagation skeleton. Finally, process all similar candidate edges according to three mutually exclusive branches: entity sameness, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges, and pending verification candidates. S7. Based on the ontology axioms set, perform consistency verification and logical completion on the merged graph, and propagate compliance tags along the lineage link according to the three-state model of original sensitivity level, inherited risk level, and desensitized shareable level, to generate a governance-level lineage knowledge graph and consistency report. S8. In response to an external audit request, based on the governance-level lineage knowledge graph, perform sensitive data flow audit, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and output the audit results. S9. Generate structured governance strategy instructions based on the audit results and issue them to external execution components. At the same time, update the graph parameters and ontology rules in a versioned manner based on manual verification feedback to form a rollback-capable full-link closed-loop governance.

2. The method according to claim 1, characterized in that, In step S2, the lineage probe adopts a bypass observer working mode, which only collects field references, runtime modes, transformation expressions and parameter class metadata, without reading actual business data values ​​or modifying data content and business logic; For scenarios where there are no source code components or manually imported classes cannot parse and transform expressions, the probe records the input and output field patterns and source identifiers, marks the operator type as unknown, and leaves it for subsequent inference and completion.

3. The method according to claim 1, characterized in that, In step S5, the four components of multidimensional semantic similarity are as follows: (1) Name similarity: After normalizing the field name and the annotation, take the larger value between the normalized edit distance similarity and the word vector cosine similarity, with a value range of 0 to 1; (2) Type compatibility: The value is 1 when the data types of the two fields are completely consistent, a preset constant is taken when lossless conversion is possible, and the value is 0 when the types are incompatible; (3) Value distribution similarity: Based on the value distribution histograms of the two fields, calculate 1 minus the Jensen-Shannon divergence of the two distributions; (4) Structural context similarity: Take the Jaccard coefficient of the set of adjacent fields along the association relationship of each field.

4. The method according to claim 1, characterized in that, In step S6, the topological time rank is determined by comparison in the following priority order: (1) External source table fields that are only used as input and never as operator output are assigned the minimum rank and sorted only by the time the data arrives; (2) The fields generated by the operator are compared in sequence with the scheduling batch number of the job, the dependency level of the job in the dependency graph, the topology number of the field in the job, and the event occurrence time. If the first item is the same, the second item is compared. (3) When the two fields belong to no common scheduling spectrum and the job dependency graph is unreachable, they are judged to be incomparable in rank, and the temporary direction is marked as undetermined.

5. The method according to claim 1, characterized in that, In step S6, the confidence iterative propagation with restart mechanism specifically involves: The initial confidence of each candidate edge is the seed value after normalizing the comprehensive similarity of that candidate edge; if two candidate edges share upstream and downstream fields or are reachable on the explicit lineage graph, they mutually confirm each other and increase the confidence; isolated highly similar candidates cannot obtain topological confirmation, and their confidence is suppressed. In each iteration, the new confidence of each candidate edge is the weighted average of its initial seed value and the propagation value of the associated candidate edge, and the weight of the two is controlled by the restart coefficient. The iteration process continues until the change in confidence between two adjacent rounds is less than the convergence threshold, or the preset maximum number of iterations is reached.

6. The method according to claim 1, characterized in that, In step S6, the decision rule for the three mutually exclusive branches is as follows: (1) Entity same branch: If the candidate confidence reaches the preset confidence threshold, meets the basic conditions of synonym identification with consistent type, highly overlapping value distribution, and belonging to different namespaces, and hits at least one strong piece of evidence and does not trigger any rejection conditions, an undirected same relation edge or a directed alias relation edge is established, and no derived relation edge is established. The strong evidence includes controlled semantic label consistency, consistent sample fingerprint matching, and registration mapping corroboration; the rejection conditions include semantic label mutual exclusion, value domain rule conflict, fingerprint matching rate not meeting the standard, and the same entity being distinguished into different roles. (2) Directed Derivation Branch: If the conditions of the same branch of the entity are not met, but the temporary direction can be determined, the confidence level reaches the threshold, the type compatibility check is passed, and the anti-loop verification is passed, a directed derivation relationship edge is established according to the temporary direction, marked as an inference edge, and the confidence level, similarity component and verification evidence are retained. (3) Branch to be verified: Candidates that do not meet the above two branch conditions are marked as to be verified and output to the manual verification queue. No formal bloodline edge is established, and the failure conditions and corresponding evidence are recorded.

7. The method according to claim 1, characterized in that, In step S7, the consistency check and logical completion specifically include: (1) Field-level transitive closure completion: Only the transitive closure between fields is calculated to form a reachable relationship that can be directly queried; (2) Conflict detection and resolution: Detect conflicts of cycle dependency and type compatibility violation, and resolve them according to the principle of explicit edges taking precedence over implicit edges and high confidence taking precedence over low confidence. (3) Matching of operators to be attributed: Match scores according to applicable scope, and associate the operators to be attributed with the business rules with the highest scores that exceed the threshold; (4) Entity alignment and merging: The same relation is merged into the same logical field according to the equivalent closure, and all original physical field nodes are retained to support cross-system tracing.

8. The method according to claim 1, characterized in that, In step S8, the direction of the derived relation edge is defined as the output field pointing to the source field; The upstream root cause tracing is performed by forward traversal along the edge, and the downstream impact surface analysis and sensitive flow audit are performed by backward traversal along the edge. The query results are hierarchically labeled into three categories: explicit lineage, inferred lineage, and candidates to be verified.

9. The method according to claim 1, characterized in that, In step S9, the versioning method update specifically includes: The confirmation and denial results of manual verification are used as positive and negative samples, and the parameter set is fine-tuned online with a bounded step size. All inference edges, parameter updates and ontology updates are recorded with version number, status and effective time. Manually denied inference edges are marked as revoked rather than physically deleted, and historical traceability and abnormal rollback are supported. Manually confirmed business rules are associated and written back to the operator node, and newly identified operator patterns are added to the operator type library.

10. A data lineage end-to-end automatic tracking and management system, characterized in that, include: The ontology modeling module is used to define ontology classes and relationships oriented towards data lineage, construct lineage ontology, and define ontology axioms; it instantiates externally input data asset metadata into data entity nodes and field nodes, structures externally input business rule documents into business rule nodes, completes node instance initialization, and outputs the initial node set, operator type library, and business rule node set; The probe injection module is used to parse the externally input ETL / ELT job into a directed acyclic graph and assign a topology number to each operation node according to the operator type library and the business rule node set. It matches the operator type of each operation node according to the operator type library and injects lineage probes on the input and output sides of each operator in a bypass observer manner. At the same time, it parses the scheduling batch number and job dependency level of the job and generates a probe registry and a directed acyclic graph of the job structure. The semantic capture module is used to collect field-level transformation semantics through the lineage probe during job execution and generate a transformation semantic event stream; The explicit lineage construction module is used to standardize each transformation event into a field-transformation-business rule triple based on the transformation semantic event stream and the lineage ontology, and write it into the knowledge graph to generate an explicit lineage graph. The similar candidate generation module is used to calculate multidimensional semantic similarity for field pairs that are cross-job, cross-source and have no explicit direct connection based on all field nodes and their attributes and structural context in the explicit lineage graph, retain field pairs whose comprehensive similarity reaches a preset similarity threshold, and generate a set of similar candidate edges. The implicit lineage inference module is used to first pre-calculate the temporary direction for each of the similar candidate edges according to the topological temporal rank, and then perform confidence iterative propagation with a restart mechanism using the explicit lineage graph as the propagation skeleton; finally, all similar candidate edges are processed according to three mutually exclusive branches: entity sameness, directed derivation, and pending verification, to generate a merged graph containing explicit lineage edges, entity aligned edges, implicit lineage edges and pending verification candidates; The governance-level graph solidification module is used to perform consistency verification and logical completion on the merged graph based on the ontology axiom set, and to propagate compliance tags along the lineage link according to the original sensitive level, inherited risk level, and desensitized shareable level three-state model to generate a governance-level lineage knowledge graph and consistency report. The compliance audit analysis module is used to respond to external audit requests, and based on the governance-level lineage knowledge graph, to perform sensitive data flow audits, root cause analysis, impact analysis, and governance gap detection using the SPARQL graph query language, and output audit results. The closed-loop governance feedback module is used to generate structured governance strategy instructions based on the audit results and issue them to external execution components. At the same time, it updates the graph parameters and ontology rules in a versioned manner based on manual verification feedback, forming a rollback-capable full-link closed-loop governance.

Images