Metadata management intelligent agent construction method and system for data weaving

Abstract

The application belongs to the technical field of big data and data weaving, and specifically discloses a metadata management intelligent agent construction method for data weaving. The method comprises the following steps: structuring management rules and performing semantic annotation to form a knowledge unit, obtaining a semantic metadata object to be managed and combining multi-dimensional features to construct a management context description object, inputting a prompt template based on the knowledge unit and the context into a large language model for management reasoning and conflict resolution, analyzing output results to generate management decisions and management explanation information and storing the management decisions and the management explanation information in association, triggering an action according to the decisions to form an audit record and a task to be confirmed manually, and finally collecting execution results and feedback information to dynamically optimize the knowledge unit or the prompt template. The application also discloses a corresponding system. The method and the system realize intelligent reasoning of a large language model and an automatic management closed loop, effectively improve the problem of complex rule conflicts, and improve the explainability and intelligent control level of metadata management.

Description

Technical Field

[0001] This application relates to the fields of big data and data weaving technology, and more specifically, to a method and system for constructing a metadata governance intelligent agent oriented towards data weaving. Background Technology

[0002] In a data weaving system, metadata not only serves to describe and semantically represent data assets, but also directly impacts data quality, reliability, compliance, security, and serviceability. As enterprise data scales continue to expand, metadata is growing exponentially in both quantity and complexity. Traditional methods based on manually formulated rules and approved changes are no longer sufficient for quality verification and lifecycle management. Especially in large language model applications, metadata has become a crucial foundational asset supporting semantic understanding, intelligent service orchestration, and intelligent decision-making. The industry urgently needs to build an intelligent governance mechanism with cognitive and reasoning capabilities.

[0003] The shortcomings of existing technologies mainly include: First, governance rules rely excessively on hard coding and manual maintenance, resulting in poor scalability and difficulty in adapting to new business changes; second, when faced with rule conflicts, there is a lack of semantic understanding and automatic adjudication mechanisms; third, the system usually only provides a conclusion of whether to pass or not, and the governance process lacks interpretability; fourth, the governance process is disconnected from the discovery of metadata and the use of services; and fifth, the degree of automation is low and the labor cost is high in complex compliance scenarios. Summary of the Invention

[0004] To address at least one deficiency or improvement need in the prior art, this invention provides a method and system for constructing metadata governance agents oriented towards data weaving, aiming to improve the problems of poor rule scalability and low automation in the prior art.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a method for constructing a metadata governance intelligent agent oriented towards data weaving is provided, the method comprising: S1. Structure and semantically annotate the governance rules to form governance rule knowledge units with clear applicable objects, applicable conditions and priorities, and store them in the knowledge base; S2. Obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario, and access role information of the semantic metadata object; S3. Based on the governance rule knowledge unit and the governance context description object, construct a prompt template and input it into the large language model. The large language model performs governance reasoning that includes quality judgment and compliance judgment and outputs a ruling suggestion when there is a rule conflict. The output result of the large language model is parsed and verified to obtain the governance reasoning result. S4. Generate governance decisions including passage, modification, restriction or blocking based on the governance reasoning results, simultaneously generate governance explanation information including the reasons for the decisions, and store the governance decisions and the governance explanation information together. S5. Trigger corresponding governance execution actions based on the governance decision, record the execution process and results of the governance execution actions to form an audit record, and generate tasks for manual confirmation for preset governance scenarios; S6. Collect the execution results of the governance actions and the feedback information of manual corrections, evaluate the effectiveness of the governance rule knowledge unit based on the information, and optimize and adjust the governance rule knowledge unit or the prompt template.

[0006] Further, in step S1, the step of structuring and semantically annotating the governance rules includes: S101. Extract the governance rules and use the attribute mapping framework to transform them into a standardized data structure containing constraints and expected outputs. S102. Perform semantic annotation on the standardized data structure in conjunction with the domain ontology, assign applicable condition parameters for specific business domains, and establish quantitative priorities based on business impact. S103. Encapsulate the standardized data structure containing the applicable condition parameters and the quantization priority into an independent governance rule knowledge unit, and import it into the governance rule knowledge base that supports graph query.

[0007] Further, in step S2, the step of constructing the governance context description object includes: S201. Parse the semantic metadata object to be governed, extract physical attribute features, and trace its entity semantic association network in the business domain. S202. Collect the lineage and runtime status of this metadata across systems, and identify the initial data source, downstream usage scenarios, and the role and permissions of the current user. S203. The physical attribute features, the entity semantic association network, and the runtime state are fused through a hierarchical model to generate the governance context description object in a multi-dimensional business context.

[0008] Further, in step S3, the step of constructing the prompt template input large language model includes: S301. Using the governance context description object as the retrieval condition, recall the relevant governance rule knowledge unit in the governance rule knowledge base, and extract the evaluation dimension constraint from the governance rule knowledge unit. S302. The evaluation dimension constraints are used as the task instruction layer, and the governance context description object is used as the fact layer to be processed. The template engine is used for structured assembly. S303. Add a preset output constraint declaration to the end of the assembled template, limiting the output to include a preset format of dimension evaluation, quality score and problem point enumeration, generate the prompt template and input it into the large language model.

[0009] Further, in step S3, the step of performing governance reasoning that includes quality and compliance judgments and outputting adjudication recommendations when rules conflict includes: S311. Using the large language model, compare the current features of the semantic metadata object with the governance standards in the prompt template to generate a quality score and compliance conclusion; S312. When multiple governance rule knowledge units are detected to have exclusive constraints on the same field, the large language model parses the underlying semantic differences between the conflicting parties in terms of data scope or computational logic. S313. Based on the priority of the knowledge units of the integrated conflict causes and corresponding governance rules, output a ruling suggestion containing the conflict type and resolution path, and transform the reasoning text into the structured governance reasoning result through text parsing verification.

[0010] Further, in step S4, the step of generating a governance decision that includes passing, modifying, restricting, or blocking includes: S401. Compare the structured governance reasoning results with the preset execution action mapping matrix. If each dimension meets the standard, generate a pass instruction. S402. For situations where there are specification flaws or semantic conflicts at the first preset risk level, generate a modified governance solution that includes standard definition recommendations or field splitting. S403. In cases involving compliance risks or conflicting interpretations at the second preset risk level, control instructions to restrict access or block metadata flow are generated based on the scope of impact to complete the generation of the governance decision.

[0011] Further, in step S4, the step of synchronously generating governance explanation information containing the rationale for the decision includes: S401. Track the rule trigger records during the reasoning process of the large language model, and extract the specific rule entries that lead to the generation of the current decision as the basis for governance; S402. Combining the usage scenario attributes of the governance context description object, use natural language generation technology to explain the business logic reasons for adopting the decision; S403. Package the governance basis, the relevant rules, and the business logic reasons into the governance explanation information, and associate and persist it with the corresponding governance decision in the storage engine.

[0012] Further, in step S5, the step of triggering the corresponding governance execution action includes: S501. Parse the instruction type of the governance decision and call the underlying management interface to perform tag addition, description modification or structural change in the metadata registry center; S502. For decisions involving de-identification or blocking, the access control component will issue interception rules to cut off non-compliant data service call links. S503. Monitor the receipt status during execution, collect the hash differences and timestamps of metadata versions before and after execution, and construct the audit record.

[0013] Further, in step S6, the step of evaluating the effectiveness of the governance rule knowledge unit includes: S601. Collect the adoption rate of governance decisions by each business line, and collect feedback information such as modification opinions submitted by human intervention. S602. Perform cluster analysis on the manual correction feedback information, count the specific rule clusters that frequently cause false alarms, and calculate the effectiveness score of the governance rule knowledge unit; S603. Based on the effectiveness score, identify weak links, add or delete applicable conditions of the governance rule knowledge unit in a targeted manner, or reconstruct the instruction logic of the prompt template to achieve optimization and adjustment.

[0014] According to a second aspect of the present invention, a metadata governance intelligent agent construction system for data weaving is also provided, comprising: The governance rule knowledge module is used to express and semantically annotate governance rules in a structured manner, forming governance rule knowledge units with clear applicable objects, applicable conditions and priorities, and storing them in the knowledge base; The governance object perception and context construction module is used to obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario and access role information of the semantic metadata object; The large language model governance reasoning module is used to construct a prompt template based on the governance rule knowledge unit and the governance context description object and input it into the large language model. The large language model performs governance reasoning that includes quality judgment and compliance judgment and outputs a ruling suggestion when there is a rule conflict. The output result of the large language model is parsed and verified to obtain the governance reasoning result. The governance decision generation and interpretation module is used to generate governance decisions including passage, modification, restriction or blocking based on the governance reasoning results, simultaneously generate governance interpretation information including the reasons for the decision, and store the governance decisions and the governance interpretation information together. The governance execution and audit module is used to trigger corresponding governance execution actions based on the governance decisions, record the execution process and results of the governance execution actions to form audit records, and generate tasks for manual confirmation for preset governance scenarios; The governance feedback and strategy evolution module is used to collect the execution results of the governance actions and the feedback information of manual corrections, evaluate the effectiveness of the governance rule knowledge unit based on the information, and optimize and adjust the governance rule knowledge unit or the prompt template.

[0015] According to a third aspect of the present invention, a metadata governance intelligent agent device for data weaving is also provided, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of any of the methods described above.

[0016] According to a fourth aspect of the invention, a storage medium is also provided that stores a computer program executable by an access authentication device, which, when run on the access authentication device, causes the access authentication device to perform the steps of any of the methods described above.

[0017] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program / instructions, characterized in that, when executed by a processor, the computer program / instructions implement the steps of any of the methods described above.

[0018] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: (1) The method and system of the present invention realize the leap from static configuration to intelligent reasoning of large language models. By structuring the rules into knowledge units and combining them with multi-dimensional business context, the large language model is used to automatically execute governance judgments. In particular, when the rules have exclusive constraints, the semantic differences can be deeply analyzed and intelligently output adjudication suggestions, which effectively solves the problems of poor scalability and difficulty in handling complex conflicts of traditional hard-coded rules.

[0019] (2) The method and system of the present invention improve the transparency and auditability of governance decisions. By tracing the rule trigger records in the reasoning process when generating decisions, objective evidence is extracted and combined with the scenario to automatically generate governance explanation information containing the reasons for the decision. The decision and explanation are associated and stored persistently, breaking the black box mode of traditional tools that only output the final result and reducing the cost of manual investigation.

[0020] (3) The method and system of the present invention construct an automated governance closed loop with adaptive evolution capabilities. By linking the underlying interface, governance actions are automatically executed and audit records are constructed. At the same time, business adoption rate and manual correction feedback are continuously collected for cluster analysis. Based on this, the applicable conditions of rules are dynamically optimized or the prompt template logic is reconstructed, so that the entire governance system can continuously self-calibrate and upgrade as the business evolves. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for constructing a metadata governance intelligent agent oriented to data weaving, provided in an embodiment of this application; Figure 2 A flowchart illustrating the knowledge-based governance rules provided in this application's embodiments; Figure 3 A flowchart of governance object awareness and context construction provided for embodiments of this application; Figure 4 A flowchart of large language model governance inference provided for embodiments of this application; Figure 5 A flowchart for the governance decision-making and interpretation generation provided in the embodiments of this application; Figure 6 The governance execution and audit flowchart provided in the embodiments of this application; Figure 7 A flowchart illustrating the governance feedback and strategy evolution provided in this application embodiment; Figure 8 This is a schematic diagram of the structure of a metadata governance intelligent agent system oriented towards data weaving, provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0025] In a first embodiment of the present invention, the present invention provides a method for constructing a metadata governance intelligent agent oriented towards data weaving, such as... Figure 1-8 As shown, it includes: S1. Structure and semantically annotate the governance rules to form governance rule knowledge units with clear applicable objects, applicable conditions and priorities, and store them in the knowledge base.

[0026] Furthermore, step S1, the step of structuring and semantically annotating the governance rules, specifically includes: S101. Extract the governance rules and use the attribute mapping framework to transform them into a standardized data structure containing constraints and expected outputs. S102. Combine the domain ontology to perform semantic annotation on the standardized data structure, assign applicable condition parameters for specific business domains, and establish quantitative priorities based on business impact. S103. Encapsulate the standardized data structure with applicable condition parameters and quantification priorities into an independent governance rule knowledge unit, and import it into the governance rule knowledge base that supports graph query.

[0027] like Figure 2 As shown, specifically, the process first extracts raw governance rules such as quality rules, compliance rules, and lifecycle rules from different business operations. Then, a pre-defined attribute mapping framework transforms these raw rules into a standardized data structure containing rule identifiers, constraints, and expected outputs. This attribute mapping framework uses a text parsing engine to map hard-coded configurations or natural language expressions of rules into a machine-readable unified key-value mapping set. For example, a data type restriction rule for an order amount field is transformed into a standardized entity where constraints point to specific metadata objects and the expected output requires a specific numeric data format, thereby unifying heterogeneous rules into a structured basic format.

[0028] Furthermore, semantic annotation is then performed on the standardized data structure described above, in conjunction with a pre-built business domain ontology. This process semantically anchors each rule to its corresponding business domain node, assigning it applicability parameters for specific business scenarios or data objects. After semantic annotation, a quantitative priority is established for each rule based on its business impact. This process calculates the priority score of the rules using a pre-defined weighted evaluation algorithm; the specific priority algorithm formula is as follows: ; In this formula, This represents the final quantitative priority score for a specific governance rule. (Parameter) This represents the total number of downstream related business domains affected by the semantic metadata object bound by this governance rule. (Parameter) Indicates the first The preset importance weights for each affected business domain, for example, the weight value of the core financial analysis business domain will be set significantly higher than that of the ordinary operational statistics business domain. Parameters This indicates the impact on the first party when the current governance rules are violated. A quantifiable value for the degree of data quality degradation caused by each business domain. Parameters This indicates the fundamental, critical benchmark score for the governance rule; for example, the benchmark score for data compliance and anonymization rules is consistently higher than the benchmark score for general naming conventions. Parameters With parameters This represents a fixed empirical coefficient used to adjust the proportional relationship between the overall impact of downstream business and the fundamental criticality of the rules themselves.

[0029] After priority assessment, the standardized data structure containing applicable condition parameters and quantified priority scores is further packaged and encapsulated into independent governance rule knowledge units. At this point, each knowledge unit not only contains static verification and judgment logic but also integrates dynamic business application contexts and reference weights for conflict resolution. Finally, these fully encapsulated governance rule knowledge units are imported into a governance rule knowledge base that supports graph queries. This knowledge base transforms rule entities into a relational network within a graph database storage structure, forming an applicability condition graph among rules.

[0030] The above steps upgrade the originally static and fragmented rules into structured graph data with rich business semantics and priority attributes, which greatly improves the efficiency and accuracy of subsequent large language model retrieval of contextual knowledge.

[0031] S2. Obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario, and access role information of the semantic metadata object.

[0032] Specifically, step S2, the step of constructing the governance context description object, includes: S201. Parse the semantic metadata object to be governed, extract physical attribute features, and trace its entity semantic association network in the business domain. S202. Collect the lineage and runtime status of this metadata across systems, and identify the initial data source, downstream usage scenarios, and the role and permissions of the current user. S203. The physical attribute features, entity semantic association network and runtime state are fused through a hierarchical model to generate a governance context description object in a multi-dimensional business context.

[0033] like Figure 3 As shown, specifically, the received semantic metadata object to be governed is first parsed, and the physical attribute characteristics of the object are extracted by reading the underlying data directory, including basic data dictionary information such as field names, data types, and data structures. Then, the data weaving platform is traced upwards along its architectural topology to query the business directory and establish an entity semantic association network for the metadata in a specific business domain, clarifying the macro-business entity to which it belongs and the associated upstream and downstream business indicators.

[0034] Next, the system calls application programming interfaces (APIs) across different business and computing engines to dynamically collect the metadata's lineage and real-time operational status. During this process, through deep analysis of scheduling logs and access control gateway records, it accurately identifies the initial data source, the downstream business usage scenarios involved, and the specific business role and permission level of the user initiating the access request.

[0035] Finally, the extracted physical attribute features, entity semantic association network, and runtime state are fused using a hierarchical model. To avoid severe redundancy in the context information input to the large language model due to the complexity of network nodes, a graph path decay algorithm is used to calculate the context relevance of each edge node in the entity semantic association network and lineage to perform pruning filtering. The specific formula for the context relevance calculation algorithm is as follows: ; In this formula, Indicates the first The contextual relevance score of each associated node in this governance task. Parameters This represents the baseline score of the basic business importance of the associated node in the global data asset dictionary. (Parameter) This represents the shortest physical path hops from the associated node to the target semantic metadata object to be governed in the lineage topology or semantic topology. (Parameter) A fixed penalty coefficient is used to control the exponential decay of relevance as the path distance in the graph network increases.

[0036] Network nodes with relevance scores greater than a preset retention threshold are selected and then concatenated with their basic physical attributes and current runtime state in a multi-branch tree hierarchical structure to generate a governance context description object with multi-dimensional business context.

[0037] The steps described above in this solution depict the real business context of metadata flowing across systems, providing high signal-to-noise ratio background support input for the large language model to perform subsequent judgments.

[0038] S3. Construct a prompt template input large language model based on governance rule knowledge units and governance context description objects. The large language model performs governance reasoning including quality judgment and compliance judgment and outputs adjudication suggestions when there is a rule conflict. The output results of the large language model are parsed and verified to obtain the governance reasoning results.

[0039] Specifically, step S3, the step of constructing the prompt template input large language model, includes: S301. Using the governance context description object as the retrieval condition, recall relevant governance rule knowledge units in the governance rule knowledge base, and extract evaluation dimension constraints from the governance rule knowledge units. S302. The evaluation dimension constraints are used as the task instruction layer, the governance context description objects are used as the facts to be processed, and the template engine is used for structured assembly. S303. Add a preset output constraint declaration to the end of the assembled template, limiting the output to include a preset format of dimensional evaluation, quality score and problem point enumeration, generate a prompt template and input it into the large language model.

[0040] like Figure 4 As shown, specifically, using the generated governance context description object as the retrieval condition, vector retrieval and graph path matching are performed in the governance rule knowledge base that supports graph queries to recall governance rule knowledge units highly related to the current metadata to be governed. Then, evaluation dimension constraints for this metadata are extracted from these recalled governance rule knowledge units. Using a structured template engine, the extracted evaluation dimension constraints are set as the task instruction layer guiding the behavior of the large language model, while the governance context description object containing the physical attributes and lineage status of the metadata is embedded as the fact layer to be processed for structured assembly. At the end of the assembled template, a preset output constraint declaration is added, requiring the large language model to output the judgment result according to a preset format including evaluations of various dimensions, a percentage quality score, and enumeration of problem points. The resulting prompt template is then input into the large language model.

[0041] Furthermore, step S3, which involves governance reasoning that includes quality and compliance assessments and outputting adjudication recommendations in the event of rule conflicts, includes: By using a large language model to compare the current characteristics of semantic metadata objects with the governance standards in the prompt template, a quality score and compliance conclusion are generated. When multiple governance rule knowledge units are detected to have exclusive constraints on the same field, the large language model analyzes the underlying semantic differences between the conflicting parties in terms of data scope or computational logic. By combining the priorities of the causes of conflict and the corresponding governance rule knowledge units, the system outputs adjudication recommendations that include conflict type and resolution path, and transforms the reasoning text into structured governance reasoning results through text parsing and verification.

[0042] like Figure 4 As shown, after receiving the prompt template, the large language model compares the current characteristics of the semantic metadata object to be governed with the governance standards injected in the template to generate a quantitative quality score and a qualitative compliance conclusion. If multiple governance rule knowledge units from different business lines are detected to have exclusive constraints on the same data field, a conflict analysis mechanism will be triggered. At this time, the large language model deeply analyzes the underlying semantic differences between the conflicting parties in terms of data caliber or computational logic. To generate scientific adjudication recommendations, the adjudication recommendation degree of each potential resolution path is calculated by combining the conflict causes extracted by the large language model with the priority of the corresponding governance rule knowledge units. The specific adjudication recommendation degree algorithm formula is as follows: ; In this formula, Indicates adoption of the first Several conflict rules are used to generate a final decision recommendation score for the resolution path. Parameters Representing the Quantification priority of each governance rule knowledge unit. Parameters This indicates the underlying computational logic of the large language model for the current semantic metadata object and the first... The semantic fit metric is calculated between the rules. The exponential fraction structure on the right-hand side of the formula is used to smoothly map the unbounded semantic fit metric to a normalized probability space. Parameters With parameters This is a preset weight constant used to balance the prior importance of business and the semantic similarity of the current context.

[0043] Based on the highest recommendation score, a ruling suggestion containing conflict type determination and standard resolution path is output. Finally, the feature extraction component is invoked to extract key-value pairs from the initial inference text output by the large language model using regular expressions, perform type conversion and boundary checks, and transform the natural language text into a fully structured governance inference result.

[0044] The above steps endow data governance with the cognitive reasoning ability to handle complex rule conflicts and ensure the determinism of the output data format of the large language model for downstream scheduling and transfer.

[0045] S4. Generate governance decisions that include passage, modification, restriction or blocking based on the governance reasoning results, simultaneously generate governance explanation information that includes the reasons for the decisions, and store the governance decisions and governance explanation information together.

[0046] Further, in step S4, the step of generating a governance decision that includes passage, modification, restriction, and / or blocking includes: S401. Compare the structured governance reasoning results with the preset execution action mapping matrix. If each dimension meets the standard, generate a pass instruction. S402. For situations where there are specification flaws or semantic conflicts at the first preset risk level, generate a modified governance solution that includes standard definition recommendations or field splitting. S403. In cases involving compliance risks or conflicting interpretations at the second preset risk level, control instructions to restrict access or block metadata flow are generated based on the scope of impact to complete the generation of governance decisions.

[0047] like Figure 5 Specifically, the structured governance reasoning results are read and compared item by item with a preset execution action mapping matrix. If the comparison results show that all dimensions of the semantic metadata meet the preset standards, a pass order is directly generated, allowing the metadata to circulate normally in the data weaving network and be called downstream. When the structured reasoning results indicate the presence of specification flaws or semantic conflicts, a preset risk assessment algorithm is introduced to quantitatively calculate the comprehensive risk score of the metadata to define its risk level range. The specific formula for calculating the comprehensive risk score is as follows: ; In this formula, This represents the overall risk score of the semantic metadata to be governed. (Parameters) This represents a benchmark value quantifying the degree of compliance violations or the severity of policy conflicts output by a large language model. Parameters This represents the total number of downstream application nodes affected by the data quality issue in this metadata. (Parameter) Indicates the first The importance weights of each affected downstream node are assigned, with the weight of the core financial statement node set significantly higher than that of ordinary data exploration nodes. Parameters Indicates the first Each node represents the historical call frequency for the metadata to be governed. Parameters With parameters This represents a fixed adjustment coefficient used to balance the baseline value of the model inference with the actual impact range of downstream business.

[0048] The specific handling strategy is determined based on the calculated comprehensive risk score. When the comprehensive risk score is within the first preset risk level range, the current situation is determined to be a basic specification flaw or a minor semantic conflict. The strategy component is then invoked to generate a modified governance solution containing recommended standard definitions or field splitting suggestions. When the comprehensive risk score reaches the second preset risk level threshold, the metadata is determined to involve high-risk data compliance risks or serious cross-system conflicts. At this point, based on the number of downstream impact nodes calculated cumulatively in the formula, control instructions are automatically generated to restrict access permissions for specific business roles or directly block the flow of the metadata in the underlying routing link of the data weaving system, thereby completing the instantiation of the governance decision.

[0049] Further, step S4, the step of simultaneously generating governance explanation information containing the rationale for the decision, includes: S411. Track the rule triggering records in the reasoning process of the large language model and extract the specific rule entries that lead to the current decision generation as the basis for governance. S412. Combine the usage scenario attributes of the object with the governance context description, and use natural language generation technology to explain the business logic reasons for adopting this decision; S413. Package the governance basis, relevant rules, and business logic reasons into governance explanation information, and associate and persist it with the corresponding governance decisions in the storage engine.

[0050] like Figure 5 As shown, after a governance decision is generated, a governance explanation information generation engine is simultaneously launched to ensure the interpretability of the decision-making process. The rule triggering records and attention mechanism weight distribution during the reasoning process of the large language model are traced in reverse to accurately extract the specific rule entries that led to the current decision as objective governance basis. Subsequently, combined with the specific business use case attributes extracted from the pre-constructed governance context description object, a natural language generation model is used to output the specific business logic reasons for adopting the decision. For example, a text explanation is generated clarifying how the mandatory splitting of indicators ensures the consistency of financial analysis business.

[0051] The extracted objective governance basis, the specific underlying rules involved, and the generated business logic reasons are packaged into governance explanation information with a unified data structure. Finally, the write interface of the distributed storage engine is called, and the governance explanation information is persistently stored with a strong association between the primary and foreign key mapping mechanism and the corresponding governance decision execution.

[0052] The above steps enable a seamless and automatic transformation from abstract reasoning results to concrete control instructions, while providing detailed and transparent business and rule basis for each governance action issued, reducing the workload of subsequent data auditing and manual investigation.

[0053] S5. Trigger corresponding governance execution actions based on governance decisions, record the execution process and results of governance execution actions to form audit records, and generate tasks for manual confirmation for preset governance scenarios.

[0054] Furthermore, step S5, which triggers the corresponding governance execution action, includes: S501: Parse the instruction type of the governance decision and call the underlying management interface to perform tag addition, description modification or structural change in the metadata registry; S502. For decisions involving de-identification or blocking, the access control component will issue interception rules to cut off non-compliant data service call links. S503: Monitor the receipt status during execution, collect the hash differences and timestamps of metadata versions before and after execution, and build audit logs.

[0055] like Figure 6 As shown, specifically, the system first receives and parses the governance decisions transmitted from the preceding stages, extracting the instruction type attribute. When the instruction type is identified as a modification operation, a request is made to the underlying management interface of the data weaving system via a remote procedure call protocol. This interface directly connects to the metadata registry center, performing physical-level tag appending, descriptive text replacement, or data type-based structural changes to the underlying data dictionary of the target semantic metadata object according to the instruction content.

[0056] For governance decisions regarding instructions that fall under the categories of desensitization or blocking, the enterprise architecture's global access control gateway and dynamic routing component will be coordinated. Dynamic routing interception rules will be issued to the access control gateway in real time. These rules contain a unique identifier for the target metadata and a blacklist of roles, thereby directly cutting off non-compliant data service call links at the underlying network layer or service bus layer, preventing unauthorized access to illegal data by downstream businesses.

[0057] During the aforementioned execution, the daemon deployed in the background continuously monitors the execution receipt status information returned by various underlying components. To ensure the traceability and immutability of the execution trajectory, full state snapshots of the metadata objects are collected before and after the governance action is executed, and the state change difference between the two is calculated. Based on this difference, it is determined whether the current action belongs to a high-risk operation in the preset governance scenario. The specific formula for quantifying the state change difference is as follows: ; In this formula, This indicates a score reflecting the difference in state changes before and after metadata execution. Parameters This represents the hash digest distribution distance of the full metadata state snapshot before and after execution, used to measure the magnitude of changes in the underlying binary-level data. Parameters This represents the number of metadata attribute fields that actually resulted in numerical or structural changes during this governance action. (Parameter) This indicates the total number of attribute fields contained in this semantic metadata object. (Parameter) With parameters These are the preset hash digest distance weight benchmark and attribute change ratio weight benchmark, respectively.

[0058] The calculated state change difference score is extracted. If the score exceeds a preset security threshold, the current change is determined to have triggered a high-risk control condition in a preset governance scenario. At this point, the automated release process is intercepted, and the current action is packaged into a task awaiting manual confirmation and pushed to the data specialist's approval workbench. Finally, the successful execution or pending approval receipt status, along with the quantified hash difference status before and after execution and a timestamp accurate to milliseconds, are packaged to construct a complete governance audit record with tamper-proof characteristics and stored in an immutable read-only log node.

[0059] The above steps enable the automated and precise implementation of metadata governance strategies in complex architectures, and ensure the execution security and controllability of underlying data assets by building an anti-tampering audit link and a quantitative risk early warning mechanism.

[0060] S6. Collect the execution results of governance actions and feedback information on manual corrections, evaluate the effectiveness of governance rule knowledge units based on the information, and optimize and adjust the governance rule knowledge units or prompt templates.

[0061] Furthermore, step S6, the step of evaluating the effectiveness of the governance rule knowledge unit, includes: S601. Collect the adoption rate of governance decisions by each business line, and collect feedback information such as modification opinions submitted by human intervention. S602. Perform cluster analysis on the manually corrected feedback information, count the specific rule clusters that frequently cause false alarms, and calculate the effectiveness score of the governance rule knowledge unit; S603. Based on the effectiveness score, identify weak links, add or delete applicable conditions for governance rule knowledge units, or reconstruct the instruction logic of prompt templates to achieve optimization and adjustment.

[0062] like Figure 7 As shown, specifically, an automated monitoring pipeline is first established to continuously monitor the data governance action flow status at the underlying level of each business line. By comparing the total number of automatically issued governance decisions with the number of instructions actually executed by the business lines, the adoption rate of the generated governance decisions by each business line is collected and calculated. Simultaneously, form data submitted by data specialists during manual intervention in the approval workbench is collected, and the modification comments and rejection reason tags are extracted. The above data is then uniformly summarized into structured manual correction feedback information.

[0063] Subsequently, a natural language processing engine is invoked to perform text vectorization and density clustering analysis on the collected human correction feedback. Semantically similar human rejection reasons are grouped into specific feedback semantic clusters, which are then back-mapped to the underlying governance basis that triggered the decision. This allows for the precise identification of specific rule clusters that frequently cause business misreports or are considered to increase understanding costs. Combining the above adoption rate and clustering statistics, a pre-defined effectiveness evaluation algorithm is introduced to quantitatively calculate the comprehensive effectiveness score of the target governance rule knowledge unit. The specific effectiveness score calculation formula is as follows: ; In this formula, Indicates the first A comprehensive effectiveness score for each governance rule knowledge unit. Parameters The representative is from the first The quantified value of the overall adoption rate of governance decisions generated by each governance rule knowledge unit across various business lines. Parameters This represents the total number of times the rule, derived from cluster analysis based on manually corrected feedback information, has been classified into a false alarm rule cluster. Parameters This indicates the total number of times the governance rule knowledge unit has been triggered and invoked within the current evaluation period. (Parameter) This represents the semantic severity baseline extracted from the modification comments submitted during human intervention. Parameter With parameters These represent the positive incentive coefficient for fixed adoption rate and the negative penalty coefficient for false alarm frequency, respectively.

[0064] The calculated comprehensive effectiveness score is compared with a preset health threshold. Based on this effectiveness score, weak links in the current governance system's knowledge network are accurately identified. When the effectiveness score of a specific governance rule knowledge unit falls below the preset threshold, an adaptive strategy update task is triggered to selectively add or remove applicable condition parameters within that governance rule knowledge unit. For example, frequently reporting errors from specific downstream data sources may be automatically removed from the rule's whitelist of applicable objects. If the root cause of the weakness points to cognitive biases in the large language model, a template update flow is initiated to reconstruct the task instruction logic in the prompt template input to the large language model. This optimization of the reasoning process is achieved by supplementing clear negative interception cases and boundary constraint lexical units.

[0065] The above steps construct a closed-loop dynamic evolution mechanism for data governance strategies, enabling underlying rules to self-calibrate based on real business feedback, and continuously improving the accuracy and business relevance of governance reasoning of large language models.

[0066] In a preferred embodiment, taking the data weaving platform of a large e-commerce company as an example, the system demonstrates how it can automatically discover and resolve conflicts in the definition of order amount metrics across business domains.

[0067] The system extracts governance rules from various data specification dictionaries of the enterprise and visualizes them. When the system detects that a semantic metadata object named "order_amt" has been newly registered in the business middle platform, it collects data across systems showing that it is being called by both order management and financial analysis scenarios.

[0068] The system first assembles the extracted single-system metadata context into a quality assessment prompt template, which is then input into a large language model for single-point quality verification. The structured prompts issued by the system are as follows: "You are an enterprise-level data governance expert."

[0069] The following is information from a metadata field: Field name: order_amt Field Description: Order Amount Data type: decimal(12,2) Business domain: Order management Use cases: Financial analysis, business analysis Please evaluate from the following dimensions: 1. Is the description complete? 2. Is the description ambiguous? 3. Does it conform to the naming and description specifications for monetary fields?

[0070] Please provide: the evaluation results for each dimension (pass / fail); the overall quality score (0–100); and the main problems (if any).

[0071] Please output the results in a structured format. Subsequently, during the lineage tracing, the system discovered that both the order system and the financial system had a field named `order_amt`, triggering the multi-source rule exclusive constraint. The system then assembled the contexts from both systems into a conflict analysis hint template and input it into the large language model. "You are a data standards and metrics governance expert."

[0072] The following are two metadata definitions: Metadata A: Name: order_amt; Description: Order amount (excluding tax); Source system: Order system.

[0073] Metadata B: Name: order_amt; Description: Actual payment amount for the order; Source system: Financial system.

[0074] Please determine: 1. Whether there is a semantic conflict between the two; 2. The type of conflict (clause conflict / naming conflict / no conflict); 3. The reason for the conflict.

[0075] Please output the results in a structured format. The large language model returned a structured reasoning result, identifying a conflict in tax treatment due to differences in whether tax is included and the deduction logic. The system calculated that this conflict could lead to distortion of financial statements, reaching the second preset risk level. It then assembled the analysis results into a decision generation prompt template and requested the large language model again. "You are an enterprise data governance architect."

[0076] The following are the results of the metadata governance analysis: Conflict type: Scope conflict; Field involved: order_amt; Departments using: Finance Department, Operations Department; Current impact: Inconsistency between financial statements and business analysis indicators.

[0077] Please generate a governance strategy plan, including: 1. Whether it is necessary to split metrics or fields; 2. Recommended standard definitions; 3. Subsequent governance actions (such as going offline, renaming, and version management).

[0078] Please output the results in a structured format. The system parses the governance strategy returned by the large language model and automatically generates execution instructions: in the metadata registry, the original field is split into order_amt_excl_tax (order amount excluding tax) and order_actual_paid (actual order payment amount), and the difference in hash state before and after execution is recorded to form an audit log.

[0079] After a period of execution, the system's automated monitoring pipeline collected execution feedback from various departments and assembled it into a strategy evolution hint template, which was then input into a large language model to evaluate the effectiveness of the rules and strategies. The following is feedback on metadata governance implementation: The Finance Department has confirmed the new field definitions. The operations department believes that splitting fields increases the cost of understanding them. Manual review approval rate: 85% Please analyze: 1. The advantages and disadvantages of the current governance strategy; 2. Whether the governance strategy generation logic needs to be adjusted; 3. Provide optimization suggestions. Please output the results in a structured format. The large language model analysis outputs a suggestion: create a logical view for the operations department at the data weaving service layer, hiding the underlying physical splitting logic. Based on this, the system automatically reconstructs the applicable conditions in the governance rule base. The next time a similar situation is encountered, the system will automatically attach governance actions to generate the front-end logical view, realizing a closed-loop self-evolution of the governance system.

[0080] In another embodiment of the invention, the invention provides a metadata governance intelligent agent construction system oriented towards data weaving. For example... Figure 8As shown, the system includes a governance rule knowledge module, a governance object perception and context construction module, a large language model governance reasoning module, a governance decision generation and interpretation module, a governance execution and auditing module, and a governance feedback and strategy evolution module.

[0081] The governance rule knowledge module is used to structure and semantically annotate governance rules, forming governance rule knowledge units with clear applicable objects, conditions, and priorities, and storing them in a knowledge base. The governance object perception and context construction module is used to obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario, and access role information of the semantic metadata object. The large language model governance reasoning module is used to construct a prompt template based on the governance rule knowledge unit and the governance context description object, input it into the large language model, and perform governance reasoning including quality and compliance judgments. When there is a rule conflict, it outputs a ruling suggestion, and the output results of the large language model are parsed and verified. The governance reasoning results are obtained; the governance decision generation and interpretation module is used to generate governance decisions including passage, modification, restriction or blocking based on the governance reasoning results, and simultaneously generate governance interpretation information including the reasons for the decision, and store the governance decisions and the governance interpretation information together; the governance execution and auditing module is used to trigger corresponding governance execution actions based on the governance decisions, record the execution process and results of the governance execution actions to form audit records, and generate tasks for manual confirmation for preset governance scenarios; the governance feedback and strategy evolution module is configured to collect the execution results of the governance execution actions and manual correction feedback information, evaluate the effectiveness of the governance rule knowledge unit, and optimize and adjust the governance rule knowledge unit or the prompt template.

[0082] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0083] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0084] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.

[0086] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0089] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0090] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a metadata governance intelligent agent oriented towards data weaving, characterized in that, include: S1. The governance rules are expressed in a structured manner and semantically annotated to form governance rule knowledge units with clear applicable objects, applicable conditions and priorities, and stored in the knowledge base; wherein, the structured expression of the governance rules includes: extracting the governance rules and using an attribute mapping framework to transform them into a standardized data structure containing constraints and expected outputs; the attribute mapping framework is used to map hard-coded configurations or natural language expressions of rules into a machine-readable unified key-value mapping set through a text parsing engine; S2. Obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario, and access role information of the semantic metadata object; S3. Based on governance rule knowledge units and governance context description objects, construct a prompt template input large language model. The large language model performs governance reasoning including quality and compliance judgments and outputs adjudication suggestions when rules conflict. The output results of the large language model are parsed and verified to obtain the governance reasoning results, specifically including: S311. Using a large language model, compare the current features of the semantic metadata object with the governance standards in the prompt template to generate a quality score and compliance conclusion; S312. When multiple governance rule knowledge units are detected to have exclusive constraints on the same field, the large language model analyzes the underlying semantic differences between the conflicting parties in terms of data scope or calculation logic; S313. For governance rule knowledge units that have exclusive constraints on the same field, combine the semantic fit metric calculated by the large language model between the underlying calculation logic of the current semantic metadata object and the rules of the governance rule knowledge unit, and the priority of the corresponding governance rule knowledge unit, output a ruling suggestion containing the conflict type and resolution path, and transform the reasoning text into the structured governance reasoning result through text parsing verification; S4. Generate a governance decision containing at least one of the following: pass, modify, restrict, or block, based on the governance reasoning results. Simultaneously generate governance explanation information containing the rationale for the decision, and store the governance decision and the governance explanation information in association. Specifically, this includes: S411. Track the rule trigger records during the reasoning process of the large language model and extract the specific rule entries that lead to the current decision generation as the basis for governance; S412. Combine the usage scenario attributes of the object described in the governance context and use natural language generation technology to explain the business logic reasons for adopting the decision; S413. Package the governance basis, the relevant rules and business logic reasons into governance explanation information and associate it with the corresponding governance decision in the storage engine for persistence. S5. Trigger corresponding governance execution actions based on governance decisions, record the execution process and results of governance execution actions to form audit records, and generate tasks for manual confirmation for preset governance scenarios; S6. Collect the execution results of governance actions and feedback information on manual corrections, evaluate the effectiveness of governance rule knowledge units, and optimize and adjust governance rule knowledge units and / or prompt templates.

2. The method for constructing a metadata governance intelligent agent oriented to data weaving as described in claim 1, characterized in that, After extracting the governance rules and transforming them into a standardized data structure containing constraints and expected outputs using an attribute mapping framework, the process also includes: The standardized data structure is semantically annotated using the domain ontology, applicable condition parameters for specific business domains are assigned, and a quantitative priority is established based on the business impact. The standardized data structure containing the applicable condition parameters and the quantization priority is encapsulated into an independent governance rule knowledge unit and imported into a governance rule knowledge base that supports graph query.

3. The method for constructing a metadata governance intelligent agent oriented to data weaving as described in claim 1, characterized in that, The steps for constructing the governance context description object specifically include: S201. Parse the semantic metadata object to be governed, extract physical attribute features, and trace its entity semantic association network in the business domain. S202. Collect the lineage and runtime status of this metadata across systems, and identify the initial data source, downstream usage scenarios, and the role and permissions of the current user. S203. The physical attribute features, the entity semantic association network, and the runtime state are fused through a hierarchical model to generate the governance context description object in a multi-dimensional business context.

4. The method for constructing a metadata governance intelligent agent oriented to data weaving as described in claim 1, characterized in that, The steps for constructing the prompt template input large language model specifically include: S301. Using the governance context description object as the retrieval condition, recall the relevant governance rule knowledge unit in the governance rule knowledge base, and extract the evaluation dimension constraint from the governance rule knowledge unit. S302. The evaluation dimension constraints are used as the task instruction layer, and the governance context description object is used as the fact layer to be processed. The template engine is used for structured assembly. S303. Add a preset output constraint declaration to the end of the assembled template, limiting the output to include a preset format of dimension evaluation, quality score and problem point enumeration, generate the prompt template and input it into the large language model.

5. The method for constructing a metadata governance intelligent agent oriented to data weaving as described in claim 1, characterized in that, The steps for evaluating the effectiveness of the governance rule knowledge unit specifically include: S601. Collect the adoption rate of governance decisions by each business line and collect feedback information on manual corrections submitted by human intervention. S602. Perform cluster analysis on the manual correction feedback information, count the specific rule clusters that frequently cause false alarms, and calculate the effectiveness score of the governance rule knowledge unit; S603. Based on the effectiveness score, identify weak links, add or delete the applicable conditions of the governance rule knowledge unit, and / or reconstruct the instruction logic of the prompt template to achieve optimization and adjustment.

6. A metadata governance intelligent agent construction system oriented towards data weaving, used to implement the metadata governance intelligent agent construction method oriented towards data weaving as described in any one of claims 1 to 5, characterized in that, include: The governance rule knowledge module is configured to express and semantically annotate governance rules in a structured manner, forming governance rule knowledge units with clear applicable objects, applicable conditions and priorities, and storing them in the knowledge base; The governance object perception and context construction module is configured to obtain the semantic metadata object to be governed, and construct a governance context description object based on the data source, usage scenario and access role information of the semantic metadata object; The large language model governance reasoning module is configured to construct a prompt template based on the governance rule knowledge unit and the governance context description object, input it into the large language model, perform governance reasoning including quality judgment and compliance judgment, output a ruling suggestion when there is a rule conflict, and parse and verify the output result of the large language model to obtain the governance reasoning result; The governance decision generation and interpretation module is configured to generate governance decisions including passage, modification, restriction or blocking based on the governance reasoning results, simultaneously generate governance interpretation information including the reasons for the decision, and store the governance decisions and the governance interpretation information together. The governance execution and audit module is configured to trigger corresponding governance execution actions based on the governance decisions, record the execution process and results of the governance execution actions to form audit records, and generate tasks for manual confirmation for preset governance scenarios; as well as The governance feedback and strategy evolution module is configured to collect the execution results and manual correction feedback information of the governance actions, evaluate the effectiveness of the governance rule knowledge unit, and optimize and adjust the governance rule knowledge unit or the prompt template.

Images