Intelligent closed-loop management method for multi-element data

Abstract

The application discloses a kind of intelligent closed-loop management methods for multi-element data, it is related to intelligent data management technical field, the method includes unified CodeAgent engine, standardization ToolRegistry tool registration module, multi-dimensional quality analysis module, closed-loop verification module, strategy reflection and dynamic evolution module.The application constructs the end-to-end autonomous closed loop of "diagnosis-repair / amplification-verification-reflection-optimization", realizes the dynamic scheduling of management tool by ReAct cognitive cycle mechanism, realizes the accurate detection of deep content level quality problem based on multidimensional joint evaluation, ensures the reliability of management result by forced closed-loop verification, realizes the adaptive continuous evolution of management strategy by relying on event-driven strategy reflection mechanism.The application greatly improves the precision and automation degree of multi-element data management, the detection rate of deep quality problem is greater than or equal to 85%, management efficiency is increased by more than 60%, manual intervention is reduced by 80%, and can be widely adapted to multi-industry, multi-scene multi-element data management demand.

Description

Technical Field

[0001] This invention relates to the field of intelligent data governance technology, and in particular to an intelligent closed-loop governance method for multi-source data. Background Technology

[0002] With the rapid development of enterprise-level data platforms, smart cities, and the industrial internet, the processing and application of diverse and heterogeneous data has become a core foundation for digital transformation, and various industries are increasingly reliant on high-quality data. Currently, enterprises and institutions need to handle diverse metadata of multiple types, sources, and formats, including structured, semi-structured, and unstructured data, during the data governance process. This data is the core support for business decision-making, model training, and system operation and maintenance.

[0003] Existing multi-dimensional data governance methods suffer from numerous technical limitations, becoming a key bottleneck restricting the improvement of data quality and the release of data value: First, they have weak capabilities in identifying content-level quality issues. Existing governance systems mainly rely on static rules or simple statistical features, only able to perform surface compliance checks such as format verification and null value detection. They cannot effectively detect deep content quality issues such as logical conflicts, factual deviations, information redundancy, or sensitive semantic leaks in text. Even when some solutions introduce large language models for semantic analysis, they are often limited to a single dimension and lack a multi-dimensional joint evaluation mechanism, resulting in a high rate of missed detection of deep quality issues. Second, governance strategies are static and fixed, lacking adaptive evolution capabilities. Most systems use preset rule bases or fixed prompt templates to drive the governance process. When faced with data distribution drift, new pollution patterns, or small sample vertical domain scenarios, it is difficult to dynamically adjust the strategy. Existing dynamic rule update mechanisms mostly rely on manual annotation. Alternatively, offline model training suffers from long update cycles and delayed responses, and is not linked to repair or augmentation capabilities, failing to achieve end-to-end strategy self-optimization. Third, generative augmentation is disconnected from the main governance process, lacking closed-loop verification. For data augmentation needs in small-sample scenarios, existing methods, when generating synthetic data using large language models, do not fully perceive the distribution characteristics of the original data, easily leading to semantic distortion or non-compliant structure of synthetic samples. Furthermore, the generated results are not rigorously verified before entering the business process, potentially introducing new noise and harming the performance of downstream tasks. Fourth, the toolchain has high coupling, limiting scalability and maintainability. Existing intelligent governance systems often tightly couple analysis, cleaning, and generation functions to specific large language model calling logic, lacking standardized interface abstraction. New tools are difficult to use plug and play, version management is difficult, and flexible tool combination scheduling and priority optimization cannot be supported, hindering the modular evolution and cross-scenario reuse of the system.

[0004] Industry statistics show that over 60% of business decision-making errors, substandard model performance, and system malfunctions are caused by deficiencies in data governance quality, with incidents stemming from deep-seated content-level quality issues accounting for over 70%. National policies explicitly require accelerating digital transformation across industries and improving data governance capabilities and quality levels. Existing governance methods are no longer sufficient to meet the high standards required for industry development. Therefore, developing an intelligent closed-loop governance method for diverse data with self-diagnosis, adaptive repair, and self-enhancing evolution capabilities has become an urgent need in the field of data governance. Summary of the Invention

[0005] This invention aims to address the technical problems of existing multi-source data governance methods, such as weak content-level quality identification capabilities, static and rigid governance strategies, disconnect between generation and amplification and the main process, high toolchain coupling, and lack of closed-loop evolution capabilities. It provides an intelligent closed-loop governance method for multi-source data, constructing an end-to-end autonomous closed loop of "diagnosis—repair / amplification—verification—reflection—optimization" to achieve high-precision, automated, and intelligent governance of multi-source data, providing high-quality data support for enterprise digital transformation, model training, and business decision-making.

[0006] To achieve the above objectives, this invention proposes an intelligent closed-loop governance method for multi-dimensional data, comprising a unified CodeAgent engine, a standardized ToolRegistry module, a multi-dimensional quality analysis module, a closed-loop verification module, and a strategy reflection and dynamic evolution module. The unified CodeAgent engine receives input multi-dimensional data chunks and, based on the structured quality report output by the multi-dimensional quality analysis module, dynamically schedules governance tools in the standardized ToolRegistry module through a ReAct cognitive loop mechanism, driving the entire closed-loop execution process. The standardized ToolRegistry module constructs a standardized tool interface system, enabling plug-and-play, version management, and decoupled invocation of various governance tools. The multi-dimensional quality analysis module performs content-level multi-dimensional quality assessment of the input multi-dimensional data and outputs a structured quality report as the basis for governance decisions. The closed-loop verification module performs mandatory closed-loop verification on the post-governance output data; data that fails verification is prohibited from entering the final dataset. The strategy reflection and dynamic evolution module, based on verification results, enhanced statistical information, and downstream task indicators, triggers event-driven strategy reflection, achieving adaptive optimization and continuous evolution of the governance strategy, ultimately outputting a standardized, high-quality post-governance dataset.

[0007] Furthermore, the standardized ToolRegistry module incorporates four core governance tools: data analysis tools, data cleaning tools, data generation tools, and data verification tools. All tools adhere to a unified interface specification. Data analysis tools are used to perform content-level multidimensional quality analysis of multivariate data. The input is raw data chunks, and the output is a structured quality report. Data cleaning tools are used to perform adaptive repair on data with quality defects. The input is the original data blocks and repair instructions, and the output is the cleaned data. The data generation tool is used to perform distribution-aware synthetic data augmentation for scenarios with small samples and class imbalance. The input is the target data structure and distribution features, and the output is synthetic data samples. The data validation tool is used to verify the logic and domain rationality of the regulated data. The input is the regulated data, and the output is the validation results and the overall quality score.

[0008] Furthermore, the multidimensional quality analysis module sequentially performs the following steps: multidimensional joint quality assessment, quality indicator normalization processing, and structured quality report generation: (1) Multi-dimensional quality joint assessment: Based on the large language model, a multi-task assessment system is constructed to jointly analyze the four core dimensions of text coherence, factual consistency, information density and sensitive information of the input data, so as to achieve full coverage detection of surface format and deep content quality; (2) Quality index normalization: The evaluation scores of each dimension are normalized to the [0,1] interval to provide a standardized quantitative basis for subsequent tool scheduling decisions; (3) Structured quality report generation: Generate a structured quality report containing coherence score, fact consistency score, information density score, sensitive information mark, and defect type location, and cache it in the database for subsequent process calls.

[0009] Furthermore, the unified CodeAgent engine, based on the ReAct cognitive loop mechanism, sequentially executes the steps of quality report parsing, problem type determination, tool scheduling decision-making, and execution process control: (1) Quality report parsing: Receive and parse structured quality reports, identify quality dimensions below the preset threshold, and locate the type and severity of data defects; (2) Problem type determination: Clustering determination is performed based on defect features. If there are repairable logical errors, information redundancy, format abnormalities, etc., it is determined to be a cleaning task; if there are class imbalance, sample scarcity, cold start, etc., it is determined to be a generation task. (3) Tool scheduling decision: Based on the current tool invocation priority weight vector, the optimal governance tool and execution parameters are selected according to the principle of maximum relevance. The tool selection follows the formula: Where T is the governance action space, wt is the priority weight of the corresponding tool, and relevance is the correlation function between the tool and the quality defect; (4) Execution process control: Schedule the corresponding tools to perform governance operations, and after the tools have finished executing, force the closed-loop verification module to perform verification, forming a closed loop for the execution of a single governance operation.

[0010] Furthermore, the closed-loop verification module sequentially performs the following steps: logical contradiction detection, domain semantic rationality judgment, comprehensive quality score calculation, and verification result output: (1) Logical contradiction detection: Based on the large language model, determine whether there are self-contradictory contents in the data after governance, and output the logical compliance score logic_score; (2) Domain semantic rationality judgment: Based on the pre-trained domain classifier, judge whether the data after governance conforms to the semantic rules and business constraints of the corresponding business domain, and output the domain compliance score domain_score; (3) Calculation of comprehensive quality score: Based on the multi-dimensional quality indicators of the post-governance data, the comprehensive quality score Q is calculated by weighted summation. The calculation formula is as follows: , where α, β, and γ are preset weighting coefficients, and c', f', and i' are the scores for the coherence, factual consistency, and information density of the data after governance, respectively; (4) Verification result output: Based on the preset logical threshold Tlogic and domain threshold Tdomain, the verification result is output. The verification function is: Where Dout represents the processed data, V=1 indicates successful verification, and V=0 indicates unsuccessful verification.

[0011] Furthermore, the strategy reflection and dynamic evolution module sequentially executes the steps of feedback information aggregation, reflection trigger condition determination, strategy dynamic reconstruction, and weight adaptive optimization: (1) Feedback information aggregation: Real-time aggregation of verification results, comprehensive quality scores, enhanced statistical information, and downstream task performance indicators generated by each verification to construct a full-process feedback data flow; (2) Determination of reflection trigger conditions: Real-time evaluation of whether the strategy reflection trigger conditions are met. The trigger conditions include: the overall quality score drops by more than 10%, three consecutive verification failures, and the effect of synthetic data on downstream tasks does not reach the preset threshold. If any condition is met, the strategy reflection will be triggered. (3) Dynamic strategy reconstruction: Aggregate historical failure cases and success cases, automatically reconstruct the system prompt example set, including replacing inefficient and few sample examples, optimizing the expression of governance instructions, and injecting verification failure cases as negative samples; (4) Weight adaptive optimization: Based on the historical governance success rate statistics, update the call priority weight vector of various tools to realize end-to-end adaptive optimization of governance strategy and establish a monthly automatic parameter tuning mechanism.

[0012] Furthermore, the data generation tool performs distribution-aware synthetic data augmentation, sequentially executing the steps of original data distribution feature analysis, few-sample cue construction, constrained synthetic generation, and pre-validation filtering: (1) Analysis of the distribution characteristics of the original data: Analyze the distribution characteristics of the original data in the three core dimensions of category, difficulty and style, and construct a target distribution profile. ; (2) Construction of few-sample prompts: Based on the target distribution profile, construct domain-related few-sample prompts, and clarify the target schema, field constraints, business rules and distribution requirements of the synthetic data; (3) Constrained synthesis generation: Based on the large language model, synthesized data samples that conform to the target distribution and structural constraints are generated to ensure that the sample semantics are reasonable and the structure is compliant; (4) Pre-verification filtering: Perform pre-format and logic verification on the generated synthetic samples, filter out samples that obviously do not meet the requirements, and output the synthetic data to be finally verified.

[0013] Furthermore, the unified CodeAgent engine has a built-in retry control and degradation strategy. For data that fails verification, a maximum number of retries, Rmax, is set. If the number of retries is less than Rmax, the scheduling tool is rescheduled to adjust parameters and perform secondary governance. If the number of retries is greater than or equal to Rmax, a degradation strategy is triggered, and the data is manually reviewed, marked, or isolated and discarded.

[0014] Furthermore, the strategy reflection and dynamic evolution module has a built-in A / B canary release and automatic flow switching mechanism. The optimized governance strategy first runs in parallel in shadow mode. When the new strategy increases the F1 value of downstream tasks by ≥5% and the governance failure rate is ≤2%, it automatically switches to the mainstream strategy to ensure the safety and stability of strategy evolution.

[0015] Furthermore, the multi-data governed by this method includes structured data, semi-structured data, and unstructured text data, and supports multiple formats such as JSON, CSV, Excel, and plain text. The output governed data conforms to the data standards and specifications of the corresponding industry and can be directly applied to scenarios such as enterprise data platforms, model training, and business decision-making.

[0016] Beneficial effects The intelligent closed-loop governance method for multi-source data of the present invention has the following significant advantages compared with the prior art: This invention significantly improves the detection and repair capabilities for deep data quality issues. Through a content-level multi-dimensional quality joint evaluation system, it achieves full coverage detection across four dimensions: text coherence, factual consistency, information density, and sensitive information, overcoming the limitation of traditional rule engines that can only detect surface-level formatting problems. Combined with the ReAct cognitive loop-driven tool scheduling mechanism and mandatory closed-loop verification process, it ensures that the repair results pass dual verification of logical consistency and domain rationality. On a 10,000-item QA test set containing manually injected logical contradictions, the detection rate of logical / factual errors is ≥85%, far exceeding the ≤30% detection rate of traditional rule engines. This effectively solves the pain points of high missed detection rates and insufficient repair reliability of existing technologies for deep quality issues.

[0017] The governance strategy possesses adaptive evolution capabilities, adapting to dynamic data environments. This invention constructs an event-driven strategy reflection and dynamic evolution mechanism, which can automatically reconstruct the system prompt example set and optimize tool call priorities based on verification feedback and downstream task indicators. Continuous optimization of the governance strategy can be achieved without manual annotation and offline training. Combined with A / B canary release and automatic flow switching mechanisms, the system can evolve securely without service interruption. When faced with five unseen new data pollution patterns, the initial repair success rate within 7 days is ≥60%, significantly better than traditional static strategy solutions, solving the problems of rigid governance strategies, delayed response, and poor adaptability in existing technologies.

[0018] The data supply quality and downstream task performance in small-sample vertical domain scenarios are significantly improved. The data generation tool of this invention adopts a distribution-aware constraint generation mechanism, which can accurately match the category, difficulty, and style distribution characteristics of the original data to generate synthetic samples that conform to business schema and semantic rules. Moreover, all generated samples must pass mandatory closed-loop verification before entering the final dataset, effectively avoiding the problem of introducing new noise into the synthetic data. On three vertical domain QA datasets, after expanding the SFT model training data from 500 to 2000, the downstream task accuracy improved by ≥15% (F1-score comparison), effectively alleviating the cold start and class imbalance problems in small-sample scenarios.

[0019] The system architecture exhibits strong decoupling, significantly enhancing scalability and reusability. This invention establishes a unified tool interface specification through a standardized ToolRegistry module, enabling decoupled deployment, plug-and-play functionality, and version management of tools for data analysis, cleaning, generation, and verification. This avoids the tight coupling defects of traditional system toolchains. The system architecture features clear layering, allowing for flexible adaptation to the diverse metadata governance needs of multiple industries and scenarios, such as enterprise-level data platforms, smart cities, and the industrial internet. Simple adjustments to domain rules and ontology libraries can quickly adapt to different business scenarios, demonstrating strong cross-domain reusability potential.

[0020] The entire governance process is highly automated, significantly reducing labor costs. This invention constructs a fully autonomous closed loop of "diagnosis-repair / amplification-verification-reflection-optimization," which automates the entire process from quality inspection, tool scheduling, governance execution to strategy optimization. Compared with traditional governance methods that rely on manual rule configuration, manual verification, and manual strategy updates, manual intervention is reduced by more than 80%, and the efficiency of single-batch data governance is improved by more than 60%, which can meet the needs of batch and routine governance of large-scale, diverse, and heterogeneous data. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the overall process of the intelligent closed-loop governance method for multi-source data according to the present invention. Figure 2 This is a schematic diagram showing the detailed execution steps of each module of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0023] This invention discloses an intelligent closed-loop governance method for diverse data, applied to the governance scenario of diverse business data in an enterprise-level data platform. The data types governed include structured business form data, semi-structured log data, and unstructured customer service dialogue and business document data, covering three major vertical domains: finance, government affairs, and industry. The implementation environment is an industrial server equipped with an Intel Xeon Gold 6330 processor and 128GB of memory, running a hardware and software platform including Python, TensorFlow, PyTorch, Milvus vector database, and Redis caching database. The specific implementation steps are as follows: Step 1: Data Preprocessing and Chunking Input. Perform format parsing and chunking processing on the raw multi-dimensional input data. Structured CSV / Excel data is chunked by row, and unstructured text data is chunked by 2000 tokens. Each data chunk is attached with metadata tags such as business domain, data source, and collection time. The preprocessed data chunks are then input into the unified CodeAgent engine.

[0024] Step 2: Multi-dimensional Quality Analysis Module Execution 2.1 Multi-dimensional Joint Quality Assessment: The unified CodeAgent engine calls the data analysis tool analyze_data_quality in the standardized ToolRegistry tool registration module, constructs a multi-task prompt template based on the GPT-4o large language model, and performs parallel quality assessments on the input data in blocks, evaluating four dimensions: text coherence, factual consistency, information density, and sensitive information, achieving full coverage detection of both surface format and deep content quality; 2.2 Quality Indicator Normalization Processing: The assessment scores of each dimension are uniformly normalized to the [0,1] interval, where the factual consistency threshold is... The value Tfact is set to 0.7, the text coherence threshold Tcoh is set to 0.75, and the sensitive information flag is determined using binary judgment. If sensitive information exists, it is marked as 1, and if no sensitive information exists, it is marked as 0; 2.3 Structured quality report generation: Generate a structured quality report quality_report containing coherence_score, fact_consistency_score, info_density_score, sensitive_flag, defect type, and defect location, and cache it in the Redis database for subsequent decision-making calls by the unified CodeAgent engine.

[0025] Step 3: Unified CodeAgent Engine Tool Scheduling and Execution 3.1 Quality Report Parsing and Problem Judgment: The Unified CodeAgent engine, based on the ReAct cognitive loop mechanism, parses the quality_report, identifies quality dimensions below a preset threshold, and clusters and determines the defect type: For repairable defects such as factual errors, logical contradictions, and information redundancy, they are classified as cleaning tasks; for scenarios with small samples in a specific domain, class imbalance, or insufficient sample size, they are classified as generation tasks. 3.2 Tool Scheduling Decision: Initialize the tool call priority weight vector w, where the weight for data cleaning tools is 0.6 and the weight for data generation tools is 0.4, based on the formula... 3.3 Execution of Governance Operations: If the data cleaning tool clean_test_cases is selected, targeted repair instructions are generated based on the defect type, guiding the large language model to generate semantically preserved corrected text and outputting the cleaned data Dout; if the data generation tool generate_test_cases is selected, the distribution characteristics of the original data in the three dimensions of category, difficulty, and style are analyzed first to construct a target distribution profile, and based on this, domain-related few-sample hints are constructed to guide the large language model to generate synthetic samples that conform to the target schema and business rules, outputting the synthetic data Dout; 3.4 Forced Verification Trigger: After the governance tool is executed, the unified CodeAgent engine forces a callback to the closed-loop verification module to perform closed-loop verification on Dout, prohibiting unverified data from entering the final dataset.

[0026] Step 4: Closed-loop verification module execution. 4.1 Logical contradiction detection: Call the data verification tool `validate_test_cases` to determine if `Dout` contains contradictory content based on the large language model, outputting `logic_score`, with the logical threshold `Tlogic` set to 0.8; 4.2 Domain semantic rationality judgment: Based on the pre-trained BERT classifier for the corresponding vertical domain, determine whether `Dout` conforms to the semantic rules and constraints of the business domain, outputting `domain_score`, with the domain threshold `Tdomain` set to 0.75; 4.3 Comprehensive quality score calculation: Set weight coefficients α=0.3, β=0.4, γ=0.3, based on the formula Q=0.3. c′+0.4 f′+0.3 i′ calculates the overall quality score Q; 4.4 Validation result output: output the final validation result based on the validation function. If logic_score≥0.8 and domain_score≥0.75, the validation passes (V=1), otherwise the validation fails (V=0).

[0027] Step 5: Retry Control and Degradation Strategy Execution. Set the maximum number of retries Rmax=3 and initialize the retry counter r=0. If the verification passes (V=1), write Dout to object storage, write metadata to the Parquet file, and include it in the final governed dataset. If the verification fails (V=0) and r<3, then r=r+1, return to step 3, adjust the tool parameters or change the governance tool to perform secondary governance. If the verification fails (V=0) and r≥3, trigger the degradation strategy, mark the data block as awaiting manual review, include it in the isolation area, and do not enter the final dataset.

[0028] Step 6: Execution of the Strategy Reflection and Dynamic Evolution Module 6.1 Feedback Information Aggregation: The intelligent data governance module aggregates feedback information such as the verification results of each verification, the comprehensive quality score Q, the acceptance rate of generated samples, and the F1-score of downstream tasks in real time to build a full-process feedback data flow; 6.2 Reflection Trigger Condition Determination: Real-time evaluation of trigger conditions. When any of the following occurs, the strategy reflection is immediately triggered: a drop of more than 10% in the overall quality score, three consecutive verification failures, or the improvement of the synthesized data on downstream tasks by less than 2%. 6.3 Dynamic strategy reconstruction: Aggregate historical failure and success cases, automatically reconstruct the system prompt example set, replace inefficient few sample examples, optimize the expression of governance instructions, and inject typical cases of recent verification failures as negative samples; 6.4 Weight Adaptive Optimization: Based on the success rate statistics of nearly 1000 governance attempts, the tool call priority weight vector w is updated, a monthly automatic parameter tuning mechanism is established, and the threshold parameters of each dimension are optimized. 6.5 Canary Release and Automatic Stream Switching: The optimized strategy is first run in parallel in shadow mode, and the running effect is continuously monitored. When the new strategy increases the F1 value of downstream tasks by ≥5% and the governance failure rate is ≤2%, it automatically switches to the mainstream strategy to ensure stable system operation.

[0029] Step 7: Output the governance results. The dataset that has undergone closed-loop governance will be output according to the data standards and specifications of the corresponding industry. At the same time, a data governance report will be generated, which will record information such as the governance process, parameter settings, quality changes, conflict resolution results, and strategy optimization, providing high-quality data support for subsequent business applications and model training.

[0030] The implementation results of this embodiment show that the intelligent closed-loop governance method for multi-data types of the present invention achieves a logic / fact error detection rate of 87.2%, a deep quality problem repair success rate of 82.5%, a first-time repair success rate of 64.8% within 7 days in the face of new types of data pollution, a 16.3% improvement in downstream task accuracy in small sample scenarios, a 68% improvement in overall governance efficiency compared to traditional methods, and an 83% reduction in manual intervention. It fully meets the high standards of enterprise-level multi-data governance and can be quickly adapted to the governance needs of multiple fields such as government affairs, industry, and finance.

[0031] The embodiments of the present invention can be adjusted according to actual needs. For example, for different business domains, the quality assessment dimensions and thresholds can be adjusted, the domain classifier can be optimized, and the system prompt example set can be updated. For data governance scenarios of different scales, the data segmentation rules, the maximum number of retries, and the policy reflection trigger conditions can be adjusted, which has good flexibility and adaptability.

[0032] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0033] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A smart closed-loop governance method for multi-source data, characterized in that, The system comprises a unified CodeAgent engine, a standardized ToolRegistry module, a multidimensional quality analysis module, a closed-loop verification module, and a strategy reflection and dynamic evolution module. The unified CodeAgent engine receives multi-dimensional data chunks as input and, based on the structured quality report output by the multidimensional quality analysis module, dynamically schedules governance tools in the standardized ToolRegistry module through the ReAct cognitive loop mechanism, driving a closed-loop execution throughout the entire process. The standardized ToolRegistry module constructs a standardized tool interface system, enabling plug-and-play, version management, and decoupled invocation of various governance tools. The multidimensional quality analysis module performs content-level multidimensional quality assessment of the input multi-dimensional data and outputs a structured quality report as a basis for governance decisions. The closed-loop verification module performs mandatory closed-loop verification on the post-governance output data; data that fails verification is prohibited from entering the final dataset. The strategy reflection and dynamic evolution module, based on verification results, enhanced statistical information, and downstream task indicators, triggers event-driven strategy reflection, achieving adaptive optimization and continuous evolution of the governance strategy, ultimately outputting a standardized, high-quality post-governance dataset.

2. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The standardized ToolRegistry module incorporates four core governance tools: data analysis tools, data cleaning tools, data generation tools, and data verification tools. All tools adhere to a unified interface specification. Data analysis tools are used to perform content-level multidimensional quality analysis of multivariate data. The input is raw data chunks, and the output is a structured quality report. Data cleaning tools are used to perform adaptive repair on data with quality defects. The input is the original data blocks and repair instructions, and the output is the cleaned data. The data generation tool is used to perform distribution-aware synthetic data augmentation for scenarios with small samples and class imbalance. The input is the target data structure and distribution features, and the output is synthetic data samples. The data validation tool is used to verify the logic and domain rationality of the regulated data. The input is the regulated data, and the output is the validation results and the overall quality score.

3. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The multidimensional quality analysis module sequentially performs the following steps: multidimensional joint quality assessment, quality indicator normalization processing, and structured quality report generation: (1) Multi-dimensional quality joint assessment: Based on the large language model, a multi-task assessment system is constructed to jointly analyze the four core dimensions of text coherence, factual consistency, information density and sensitive information of the input data, so as to achieve full coverage detection of surface format and deep content quality; (2) Quality index normalization: The evaluation scores of each dimension are normalized to the [0,1] interval to provide a standardized quantitative basis for subsequent tool scheduling decisions; (3) Structured quality report generation: Generate a structured quality report containing coherence score, fact consistency score, information density score, sensitive information mark, and defect type location, and cache it in the database for subsequent process calls.

4. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The unified CodeAgent engine, based on the ReAct cognitive loop mechanism, sequentially executes the steps of quality report parsing, problem type determination, tool scheduling decision-making, and execution process control: (1) Quality report parsing: Receive and parse structured quality reports, identify quality dimensions below the preset threshold, and locate the type and severity of data defects; (2) Problem type determination: Clustering determination is performed based on defect characteristics. If there are repairable logical errors, information redundancy, format abnormalities, etc., it is determined to be a cleaning task. If there are problems such as class imbalance, scarce samples, or cold start, it is determined to be a generation task. (3) Tool scheduling decision: Based on the current tool invocation priority weight vector, the optimal governance tool and execution parameters are selected according to the principle of maximum relevance. The tool selection follows the formula: Where T is the governance action space, wt is the priority weight of the corresponding tool, and relevance is the correlation function between the tool and the quality defect; (4) Execution process control: Schedule the corresponding tools to perform governance operations, and after the tools have finished executing, force the closed-loop verification module to perform verification, forming a closed loop for the execution of a single governance operation.

5. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The closed-loop verification module sequentially performs the following steps: logical contradiction detection, domain semantic rationality judgment, comprehensive quality score calculation, and verification result output. (1) Logical contradiction detection: Based on the large language model, determine whether there are self-contradictory contents in the data after governance, and output the logical compliance score logic_score; (2) Domain semantic rationality judgment: Based on the pre-trained domain classifier, judge whether the data after governance conforms to the semantic rules and business constraints of the corresponding business domain, and output the domain compliance score domain_score; (3) Calculation of comprehensive quality score: Based on the multi-dimensional quality indicators of the post-governance data, the comprehensive quality score Q is calculated by weighted summation. The calculation formula is as follows: , where α, β, and γ are preset weighting coefficients, and c', f', and i' are the scores for the coherence, factual consistency, and information density of the data after governance, respectively; (4) Verification result output: Based on the preset logical threshold Tlogic and domain threshold Tdomain, the verification result is output. The verification function is: Where Dout represents the processed data, V=1 indicates successful verification, and V=0 indicates unsuccessful verification.

6. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The strategy reflection and dynamic evolution module sequentially executes the following steps: feedback information aggregation, reflection trigger condition determination, strategy dynamic reconstruction, and weight adaptive optimization. (1) Feedback information aggregation: Real-time aggregation of verification results, comprehensive quality scores, enhanced statistical information, and downstream task performance indicators generated by each verification to construct a full-process feedback data flow; (2) Determination of reflection trigger conditions: Real-time evaluation of whether the strategy reflection trigger conditions are met. The trigger conditions include: the overall quality score drops by more than 10%, three consecutive verification failures, and the effect of synthetic data on downstream tasks does not reach the preset threshold. If any condition is met, the strategy reflection will be triggered. (3) Dynamic strategy reconstruction: Aggregate historical failure cases and success cases, automatically reconstruct the system prompt example set, including replacing inefficient and few sample examples, optimizing the expression of governance instructions, and injecting verification failure cases as negative samples; (4) Weight adaptive optimization: Based on the historical governance success rate statistics, update the call priority weight vector of various tools to realize end-to-end adaptive optimization of governance strategy and establish a monthly automatic parameter tuning mechanism.

7. The intelligent closed-loop governance method for multi-source data according to claim 2, characterized in that, The data generation tool performs distribution-aware synthetic data augmentation, sequentially executing the following steps: original data distribution feature analysis, few-sample prompt construction, constrained synthetic generation, and pre-validation filtering. (1) Analysis of the distribution characteristics of the original data: Analyze the distribution characteristics of the original data in the three core dimensions of category, difficulty and style, and construct a target distribution profile. ; (2) Construction of few-sample prompts: Based on the target distribution profile, construct domain-related few-sample prompts, and clarify the target schema, field constraints, business rules and distribution requirements of the synthetic data; (3) Constrained synthesis generation: Based on the large language model, synthesized data samples that conform to the target distribution and structural constraints are generated to ensure that the sample semantics are reasonable and the structure is compliant; (4) Pre-verification filtering: Perform pre-format and logic verification on the generated synthetic samples, filter out samples that obviously do not meet the requirements, and output the synthetic data to be finally verified.

8. The intelligent closed-loop governance method for multi-source data according to claim 4, characterized in that, The unified CodeAgent engine has built-in retry control and degradation strategies. For data that fails verification, a maximum number of retries, Rmax, is set. If the number of retries is less than Rmax, the scheduling tool is rescheduled to adjust parameters and perform secondary governance. If the number of retries is greater than or equal to Rmax, a degradation strategy is triggered, and the data is manually reviewed, marked, or isolated and discarded.

9. The intelligent closed-loop governance method for multi-source data according to claim 6, characterized in that, The strategy reflection and dynamic evolution module has a built-in A / B canary release and automatic flow switching mechanism. The optimized governance strategy first runs in parallel in shadow mode. When the new strategy increases the F1 value of downstream tasks by ≥5% and the governance failure rate is ≤2%, it automatically switches to the mainstream strategy to ensure the safety and stability of strategy evolution.

10. The intelligent closed-loop governance method for multi-source data according to claim 1, characterized in that, The multi-data governance includes structured data, semi-structured data, and unstructured text data, supporting multiple formats such as JSON, CSV, Excel, and plain text. The output governance data conforms to the data standards and specifications of the corresponding industry and can be directly applied to enterprise data platforms, model training, and business decision-making scenarios.

Images