Data analysis and governance system based on adaptive rule engine and model fusion

Abstract

The application provides a data analysis and governance system based on adaptive rule engine and model fusion. The system comprises a data portrait construction module, an intelligent rule engine module, a model fusion decision module and a feedback optimization module. The data portrait construction module is used for collecting multi-source data and performing feature extraction, generating structured data portraits and storing the data portraits in a portrait index library. The intelligent rule engine module is used for performing data quality diagnosis and problem classification based on the data portraits and a static rule library and a dynamic rule generator. The model fusion decision module is used for recommending a fusion governance strategy based on the classification results, analyzing and modeling the data after governance, and outputting multi-dimensional index data of the analysis model. The feedback optimization module is used for driving the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional index data. The system provided by the application can automatically identify multi-source data quality problems and generate a governance scheme by constructing a three-layer system of data portrait + intelligent rule engine + model fusion decision, thereby realizing integration of analysis and governance.

Description

Technical Field

[0001] This invention relates to the fields of data analysis and data governance technology, and in particular to a data analysis and governance system and method based on adaptive rule engine and model fusion. Background Technology

[0002] In the context of the rapid development of big data and artificial intelligence technologies, enterprises and institutions have accumulated massive amounts of structured and unstructured data. These data vary significantly in terms of source, format, quality, and granularity, often leading to the following problems in the analysis and decision-making process: 1. Severe data silos and redundancy: Data standards are not consistent between different systems, resulting in problems such as duplication, conflicts, and version inconsistencies; 2. Data quality is difficult to guarantee: Traditional governance tools rely solely on static rule bases and cannot be flexibly adjusted for dynamic data scenarios; 3. Fragmented application of analytical models: The separation of data analysis and governance processes leads to high costs in connecting data cleaning, modeling, and application. 4. High reliance on manual rules: There is a lack of intelligent mechanisms to automatically identify data anomalies and optimize governance strategies.

[0003] In summary, existing technologies mostly focus on single aspects such as "ETL process optimization," "metadata management," or "data quality inspection," lacking a comprehensive system that can dynamically link analysis models and governance rules. Summary of the Invention

[0004] The purpose of this invention is to provide a data analysis and governance system and method based on the fusion of an adaptive rule engine and a model, aiming to solve problems such as the fragmentation of data analysis and governance processes, rigid rule configuration, low data quality, and delayed governance feedback in the prior art.

[0005] This invention provides a data analysis and governance system based on adaptive rule engine and model fusion, comprising: The data profile building module, connected to the intelligent rule engine module, is used to collect multi-source data, extract features from the multi-source data, generate structured data profiles and store them in the profile index library, and transmit the data profiles to the intelligent rule engine module. The intelligent rule engine module is connected to the data profile building module, the model fusion decision module and the feedback optimization module. It is used to perform data quality diagnosis and problem classification based on the data profile and based on the static rule base and dynamic rule generator, and send the classification results to the model fusion decision module. The model fusion decision module is connected to the intelligent rule engine module and the feedback optimization module. It is used to recommend and execute the corresponding fusion governance strategy based on the hierarchical results, analyze and model the governed data, and output multi-dimensional indicator data of the analysis model. The feedback optimization module, connected to the model fusion decision module and the intelligent rule engine module, is used to drive the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional indicator data.

[0006] This invention provides a data analysis and governance method based on the fusion of an adaptive rule engine and a model, comprising: The data profiling module collects multi-source data, extracts features from the multi-source data, generates a structured data profile and stores it in the profile index library, and then transmits the data profile to the intelligent rule engine module. The intelligent rule engine module performs data quality diagnosis and problem classification based on the data profile and on the static rule base and dynamic rule generator, and sends the classification results to the model fusion decision module. Based on the hierarchical results, the model fusion decision module recommends and executes corresponding fusion governance strategies, analyzes and models the governed data, and outputs multi-dimensional indicator data of the analysis model. The feedback optimization module drives the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional indicator data.

[0007] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described data analysis and governance method based on adaptive rule engine and model fusion.

[0008] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described data analysis and governance method based on adaptive rule engine and model fusion.

[0009] The following beneficial effects can be achieved by adopting the embodiments of the present invention: The embodiments of the present invention provide an adaptive, intelligent, and closed-loop data analysis and governance system. This system, by constructing a three-layer system of "data profiling + intelligent rule engine + model fusion decision", can automatically identify multi-source data quality problems and generate governance solutions; can automatically select the optimal modeling strategy according to the analysis task; can dynamically update the data profile after governance is executed, and achieve self-evolution; and supports visual analysis and quality tracking, realizing the integration of analysis and governance. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of a data analysis and governance system based on adaptive rule engine and model fusion according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall process of an embodiment of the present invention; Figure 3 This is a flowchart of a data analysis and governance method based on adaptive rule engine and model fusion according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] System Implementation Examples According to embodiments of the present invention, a data analysis and governance system based on the fusion of an adaptive rule engine and a model is provided. Figure 1 This is a schematic diagram of a data analysis and governance system based on the fusion of an adaptive rule engine and a model, as described in an embodiment of the present invention. Figure 1 As shown, the data analysis and governance system based on adaptive rule engine and model fusion according to an embodiment of the present invention specifically includes: The data profiling module 10, connected to the intelligent rule engine module, is used to collect multi-source data, extract features from the multi-source data, generate structured data profiles and store them in a profile index library, and transmit the data profiles to the intelligent rule engine module. Specifically, it is used for: Extract mean, variance, interquartile range, skewness, and kurtosis from numerical fields; Extract character diversity index, encoding confidence, and topic distribution from text fields; Perform periodicity detection and mutation point identification on time series fields.

[0014] The intelligent rule engine module 12 is connected to the data profile construction module, the model fusion decision module and the feedback optimization module. It is used to perform data quality diagnosis and problem classification based on the data profile and based on the static rule base and dynamic rule generator, and send the classification results to the model fusion decision module. The dynamic rule generator includes a triggering unit, a rule mining unit, a rule structuring unit, and a rule lifecycle management unit connected in sequence. The triggering unit is activated when the density of anomalies in the data profile exceeds a threshold, or when a new cluster of anomalous data is discovered by a clustering algorithm that is not covered by existing rules. The rule mining unit is used to perform feature analysis on the new abnormal data cluster and extract common feature patterns; The rule structuring unit is used to automatically compile the common feature patterns into new rules; The rule lifecycle management unit is used to promote, demote, or eliminate new rules based on their contribution to the performance of the analysis model after execution.

[0015] The model fusion decision module 14 is connected to the intelligent rule engine module and the feedback optimization module. It is used to recommend and execute the corresponding fusion governance strategy based on the hierarchical results, analyze and model the governed data, and output multi-dimensional indicator data of the analysis model. Feedback optimization module 16, connected to the model fusion decision module and the intelligent rule engine module, is used to drive the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional indicator data, specifically for: Based on the prediction residual information in the multidimensional index data, a subset of data whose prediction residuals are consistently higher than the threshold is identified as a suspected set of hidden dirty data. The contribution of the suspected hidden dirty data set is calculated using model interpretability techniques to identify one or more feature fields that contribute the most to high residuals. The dynamic rule generator is driven to create new or optimize existing detection rules for the located feature fields; Specifically, driving the dynamic rule generator to create new detection rules for the located feature fields includes: The statistical distribution information of the located feature fields in the data profile is traced, and the deviation of the statistical distribution between them and normal data samples is identified. New statistical detection rules are generated based on the identified statistical distribution deviation.

[0016] The system further includes: The continuous learning module, connected to the intelligent rule engine module and the feedback optimization module, is used to evaluate the long-term effects of governance and analysis strategies through a reinforcement learning mechanism using a preset reward function, and to dynamically adjust system parameters based on the policy gradient method. The visualization module, connected to the data profiling module, intelligent rule engine module, and model fusion decision module, provides users with a multi-dimensional visualization dashboard that displays real-time quality score curves, anomaly trend heatmaps, and governance execution success rates, and supports correlation analysis between data quality scores and business indicators. The security management module is connected to the data profiling module, intelligent rule engine module, model fusion decision module, and feedback optimization module, and is used to provide at least one of the following security supports for each module in the system: unified authentication, permission isolation, operation log auditing, and data anonymization.

[0017] The following describes in detail the above-mentioned technical solutions of the present invention with reference to the specific circumstances of the data analysis and governance system based on adaptive rule engine and model fusion in the embodiments of the present invention.

[0018] This invention proposes a data analysis and data governance system based on the fusion of an adaptive rule engine and an intelligent model. The system comprises five main parts: a data profiling layer, an intelligent rule layer, a model fusion layer, a feedback optimization layer, and a visualization decision-making layer. Through multi-source data access, dynamic profiling construction, rule engine-driven governance, intelligent model-assisted analysis, and a closed-loop feedback learning mechanism, the system achieves integrated, intelligent, and self-evolving data governance and analysis. The specific execution flow is as follows: Figure 2 As shown: Step 1: Multi-source data collection and profile building The system incorporates asynchronous queues and a schema self-checking mechanism during the data acquisition phase to automatically identify characteristics such as field type, time attribute, missing rate, and abnormal distribution. It supports multiple data types, including relational databases, time-series databases, object storage, API streams, and log streams, and features high-concurrency asynchronous acquisition and format standardization capabilities.

[0019] Numerical field extraction: mean, variance, interquartile range, skewness, kurtosis; Text field extraction: character diversity index, encoding confidence, topic distribution; Time series field extraction: periodicity detection and mutation point identification. The profile results are stored in the Profile Index DB, providing a foundation for subsequent rule selection and model training.

[0020] Step 2: Initialize the intelligent rule engine The system combines static rules and dynamic learning models to form a "dual-layer engine". It uses feature statistics and embedding learning algorithms to generate profiles of fields, including information such as numerical distribution, time span, outlier density, and semantic labels. The data profiles output by this module serve as key inputs for subsequent rule matching and model learning.

[0021] Static rule base: Covers common quality checks (null values, format, uniqueness, foreign key consistency, etc.) as a basic guarantee; The dynamic rule generator continuously optimizes parameters through supervised learning to achieve a "rule system that evolves with the data"; its workflow is as follows: 1. Triggering mechanism: The generator is activated when the "outlier density" metric in the data profile exceeds the threshold, or when the clustering module discovers a new high-density outlier data cluster that cannot be covered by any existing rules. 2. Rule mining: The generator performs feature analysis on the abnormal cluster, for example, extracting its common features as "the value of field X is concentrated in the interval (a, b)" and "90% of the records are generated within the time period of 18:00-24:00 of the time field T". 3. Rule structuring: The system automatically compiles the above feature patterns into a new, executable DSL rule; 4. Rule Lifecycle Management: Newly generated rules enter a "trial period," during which their execution effectiveness is continuously monitored. The feedback optimization layer evaluates the effectiveness of a rule based on its contribution to improving the accuracy of the analysis model after the data it captures has been processed (i.e., the value of the reward function R). If the contribution remains positive, the rule is converted into a formal rule; otherwise, it is automatically downgraded or eliminated.

[0022] The engine supports DSL (Domain-Specific Language) rule definition syntax, allowing users to customize conditional expressions and repair actions.

[0023] Step 3: Data Quality Diagnosis and Problem Classification It features a built-in static rule base and dynamic rule model, supporting rule orchestration based on DSL syntax. The engine identifies abnormal data patterns through clustering algorithms (such as DBSCAN) and automatically selects the appropriate rule set using feature similarity calculations.

[0024] By executing the set of rules, the system calculates the data quality score Q: ; Where C represents completeness, F represents consistency, and U represents uniqueness, with weighting coefficients... It can be dynamically adjusted.

[0025] The system categorizes problems into three levels: "blocking level," "alarm level," and "notification level." A pool of problem samples is generated based on the clustering results for subsequent model learning.

[0026] Step 4: Recommendation and Automated Execution of Governance Strategies By leveraging knowledge graphs and historical governance samples, the system uses the LightGBM model to predict optimal governance methods (such as filling, deletion, rule revision, and regular expression reconstruction). It employs models like LightGBM, XGBoost, and LSTM to predict and recommend governance strategies, enabling intelligent repair, filling, or structured transformation of different data types.

[0027] If the confidence level is ≥0.8, it will be executed automatically; otherwise, it will enter the manual approval queue; the execution record will be written to the governance log and audit chain.

[0028] Step 5: Integration of Data Analysis and Modeling The rule engine parameters are dynamically adjusted by analyzing metrics such as the model's prediction residuals and feature importance, forming a closed-loop feedback mechanism. After processing, the data automatically flows into the analysis module, where the system selects an algorithm based on the task type. Clustering task → K-Means / DBSCAN; Prediction task → XGBoost / LSTM; Statistical tasks → ARIMA / Prophet; The analysis module and the governance module share feature metadata, enabling "data to be cleaned once and reused multiple times".

[0029] Step 6: Reverse governance mechanism for hidden dirty data based on residual analysis A reverse feedback channel is established from "analysis results" to "governance rules" to address the technical challenge of traditional governance tools being unable to handle hidden dirty data that "complies with rules but is actually erroneous." For example, when the model residual exceeds a threshold, the system automatically identifies potential "hidden dirty data" and feeds it back to the rule engine, dynamically revising the detection logic. Specifically: 1. Hidden Dirty Data Identification: The prediction residuals of the system monitoring and analysis model (such as XGBoost). When it is found that the prediction residuals of a certain data subset (such as those from a specific data source S) are consistently significantly higher than the overall level, it is marked as a suspected set of "hidden dirty data".

[0030] 2. Root Cause Analysis and Feature Localization: The system utilizes model interpretability techniques (such as SHAP analysis) to analyze high residual samples and calculate the contribution of each input feature to the high residuals. For example, the analysis might reveal that field F has an unusually high contribution. 3. Reverse rule generation / optimization: The system traced the data profile of the suspect set and found that although field F scored highly in routine checks (such as completeness and format), its numerical distribution had statistical biases compared with low residual samples (e.g., smaller variance and abnormal quantiles).

[0031] Based on this discovery, the system will dynamically create a new, more refined statistical detection rule; This new rule has been added to the rule engine, enabling the identification and handling of such previously undetectable "hidden" problems during the governance phase the next time data flows in, preventing them from contaminating the analysis results.

[0032] Step 7: Visualization and Monitoring The system provides a multi-dimensional quality dashboard (Data Quality Dashboard) that displays: real-time quality score curves, anomaly trend heatmaps, and governance execution success rates; and supports correlation analysis between quality scores and business metrics (such as sales revenue and forecast accuracy).

[0033] Step 8: Continuous Learning and Optimization The system introduces a reinforcement learning mechanism, using long-term performance as a reward signal: ; The intelligent agent dynamically adjusts parameters through the policy gradient method to achieve cross-cycle self-optimization, enabling the governance strategy to continuously evolve with the data.

[0034] Step 9: Security and Performance Assurance Mechanism The system integrates unified authentication (OAuth 2.0) and permission isolation mechanisms, supporting operation log auditing and data anonymization. In terms of performance, asynchronous task sharding, GPU acceleration, and ElasticSearch index optimization reduce the latency for managing tens of millions of data points to the second level, significantly improving governance and analysis efficiency.

[0035] In summary, this invention provides an adaptive, intelligent, and closed-loop data analysis and governance system. This system achieves the following objectives by constructing a three-layer architecture of "data profiling + intelligent rule engine + model fusion decision-making": 1. Automatically identify multi-source data quality issues and generate remediation solutions; 2. Automatically select the optimal modeling strategy based on the analysis task; 3. Dynamically update data profiles after governance implementation to achieve self-evolution; 4. Supports visual analysis and quality tracking, achieving integrated analysis and governance.

[0036] Method Implementation Examples According to embodiments of the present invention, a data analysis and governance method based on the fusion of an adaptive rule engine and a model is provided. Figure 3 This is a flowchart of a data analysis and governance method based on adaptive rule engine and model fusion according to an embodiment of the present invention, as shown below. Figure 3 As shown, the data analysis and governance method based on adaptive rule engine and model fusion according to an embodiment of the present invention specifically includes: Step S301: Collect multi-source data through the data profile building module, extract features from the multi-source data, generate a structured data profile and store it in the profile index library, and transmit the data profile to the intelligent rule engine module. Step S302: The intelligent rule engine module performs data quality diagnosis and problem classification based on the data profile and the static rule base and dynamic rule generator, and sends the classification results to the model fusion decision module. Step S303: Based on the hierarchical results, the model fusion decision module recommends and executes the corresponding fusion governance strategy, analyzes and models the governed data, and outputs multi-dimensional indicator data of the analysis model. Step S304: The feedback optimization module drives the dynamic rule generator to create or optimize the corresponding detection rules based on the multi-dimensional index data. The method further includes: The continuous learning module uses a reinforcement learning mechanism to evaluate the long-term effects of governance and analysis strategies using a preset reward function, and dynamically adjusts system parameters based on the policy gradient method. The visualization module provides users with a multi-dimensional visualization dashboard, displaying real-time quality score curves, anomaly trend heatmaps, and governance execution success rates, and supports correlation analysis between data quality scores and business indicators; The security management module provides at least one of the following security supports for each module: unified authentication, access control isolation, operation log auditing, and data anonymization.

[0037] The embodiments of the present invention are method embodiments corresponding to the system embodiments described above. The specific operations of each step can be understood by referring to the description of the system embodiments, and will not be repeated here.

[0038] In summary, compared with the prior art, the embodiments of the present invention have the following positive effects: 1. Integrated Analysis and Governance: The system in this embodiment organically integrates the two traditionally separate processes of data analysis and data governance. Through a unified data foundation and rule engine, it achieves closed-loop management of the entire data process, from collection, cleaning, and modeling to application. Compared to traditional methods, it eliminates data transfer and redundant processing between multiple systems, significantly improving overall processing efficiency and consistency. In actual deployment, data flow latency is reduced by approximately 40%, and data availability is increased by 30%.

[0039] 2. Intelligent Rule Adaptation: The system's rule engine possesses self-creation and lifecycle management capabilities. It can automatically discover unknown and new anomalous data patterns through unsupervised clustering and structure and automatically compile them into new executable governance rules. This mechanism fundamentally distinguishes the system from existing systems that can only optimize parameters, enabling the autonomous expansion and evolution of the rule base and solving the fundamental technical challenge of static rule bases being unable to cope with dynamically changing data environments. When new data features or anomalous patterns emerge, the system can automatically identify and generate new detection and repair rules without manual configuration, thus greatly reducing the workload of rule maintenance. This feature is particularly suitable for multi-source, heterogeneous, and dynamically changing data environments.

[0040] 3. Dynamic Quality Tracking: The system constructs a data profile index library to monitor the integrity, consistency, and uniqueness of each field in real time. When data quality changes, the system can immediately issue quality alarms and automatically trigger governance tasks, achieving dynamic maintenance of data quality. Compared to traditional periodic inspection mechanisms, dynamic quality tracking can detect potential problems earlier, preventing the accumulation and spread of issues.

[0041] 4. Quantification of Governance Effects: By linking data governance results with model analysis performance (such as accuracy, residuals, and prediction bias), the system can quantify the impact of each governance action on business analysis results. For example, the decrease in the model's prediction error after missing value imputation in a field can be used as an evaluation criterion for governance effectiveness, achieving scientific and refined management of data governance.

[0042] 5. High Scalability: The system adopts a modular and microservice architecture, supporting rapid integration with third-party ETL tools, BI platforms, and visualization systems. Each functional module (such as data collection, profiling, rule engine, and model analysis) can be deployed independently and scaled horizontally, meeting the large-scale, high-concurrency requirements of enterprise-level data governance platforms. The system also supports hybrid cloud and on-premises deployment and is compatible with multiple databases and file formats.

[0043] 6. Significantly Improved Efficiency: In typical scenarios involving the governance of tens of millions of data points, this embodiment of the invention improves data processing speed by more than 45% through asynchronous task scheduling and GPU acceleration optimization; combined with rule engine and model joint detection, the error detection rate is improved by 30%. At the same time, automated execution processes reduce manual intervention, shortening the overall governance cycle from several hours to less than 30 minutes, significantly reducing operation and maintenance and manual review costs.

[0044] 7. Achieving Closed-Loop Governance of Hidden Data Issues: This invention, for the first time, achieves effective governance of "hidden dirty data." By establishing a reverse feedback loop from analytical model residuals to governance rule generation, and utilizing model interpretability techniques (such as SHAP) to accurately locate problem characteristics, the system can discover and process data that superficially conforms to all quality rules but actually leads to analytical bias. This is a capability completely lacking in traditional data governance tools, achieving an upgrade in governance dimensions from "surface quality" to "value quality."

[0045] Device Example 1 This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it performs the steps described in the method embodiment.

[0046] Device Example 2 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, performs the steps described in the method embodiment.

[0047] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data analysis and governance system based on adaptive rule engine and model fusion, characterized in that, include: The data profile building module, connected to the intelligent rule engine module, is used to collect multi-source data, extract features from the multi-source data, generate structured data profiles and store them in the profile index library, and transmit the data profiles to the intelligent rule engine module. The intelligent rule engine module is connected to the data profile building module, the model fusion decision module and the feedback optimization module. It is used to perform data quality diagnosis and problem classification based on the data profile and based on the static rule base and dynamic rule generator, and send the classification results to the model fusion decision module. The model fusion decision module is connected to the intelligent rule engine module and the feedback optimization module. It is used to recommend and execute the corresponding fusion governance strategy based on the hierarchical results, analyze and model the governed data, and output multi-dimensional indicator data of the analysis model. The feedback optimization module, connected to the model fusion decision module and the intelligent rule engine module, is used to drive the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional indicator data.

2. The system according to claim 1, characterized in that, The system further includes: The continuous learning module, connected to the intelligent rule engine module and the feedback optimization module, is used to evaluate the long-term effects of governance and analysis strategies through a reinforcement learning mechanism using a preset reward function, and to dynamically adjust system parameters based on the policy gradient method. The visualization module, connected to the data profiling module, intelligent rule engine module, and model fusion decision module, provides users with a multi-dimensional visualization dashboard that displays real-time quality score curves, anomaly trend heatmaps, and governance execution success rates, and supports correlation analysis between data quality scores and business indicators. The security management module is connected to the data profiling module, intelligent rule engine module, model fusion decision module, and feedback optimization module, and is used to provide at least one of the following security supports for each module in the system: unified authentication, permission isolation, operation log auditing, and data anonymization.

3. The system according to claim 1, characterized in that, The data profiling module is specifically used for: Extract mean, variance, interquartile range, skewness, and kurtosis from numerical fields; Extract character diversity index, encoding confidence, and topic distribution from text fields; Perform periodicity detection and mutation point identification on time series fields.

4. The system according to claim 1, characterized in that, The dynamic rule generator includes a triggering unit, a rule mining unit, a rule structuring unit, and a rule lifecycle management unit connected in sequence. The triggering unit is activated when the density of anomalies in the data profile exceeds a threshold, or when a new cluster of anomalous data is discovered by a clustering algorithm that is not covered by existing rules. The rule mining unit is used to perform feature analysis on the new abnormal data cluster and extract common feature patterns; The rule structuring unit is used to automatically compile the common feature patterns into new rules; The rule lifecycle management unit is used to promote, demote, or eliminate new rules based on their contribution to the performance of the analysis model after execution.

5. The system according to claim 1, characterized in that, The feedback optimization module is specifically used for: Based on the prediction residual information in the multidimensional index data, a subset of data whose prediction residuals are consistently higher than the threshold is identified as a suspected set of hidden dirty data. The contribution of the suspected hidden dirty data set is calculated using model interpretability techniques to identify one or more feature fields that contribute the most to high residuals. The dynamic rule generator is driven to create new or optimize existing detection rules for the located feature fields.

6. The system according to claim 5, characterized in that, The process of driving the dynamic rule generator to create new detection rules for the located feature fields specifically includes: The statistical distribution information of the located feature fields in the data profile is traced, and the deviation of the statistical distribution between them and normal data samples is identified. New statistical detection rules are generated based on the identified statistical distribution deviation.

7. A data analysis and governance method based on the fusion of an adaptive rule engine and a model, characterized in that, include: The data profiling module collects multi-source data, extracts features from the multi-source data, generates a structured data profile and stores it in the profile index library, and then transmits the data profile to the intelligent rule engine module. The intelligent rule engine module performs data quality diagnosis and problem classification based on the data profile and on the static rule base and dynamic rule generator, and sends the classification results to the model fusion decision module. Based on the hierarchical results, the model fusion decision module recommends and executes corresponding fusion governance strategies, analyzes and models the governed data, and outputs multi-dimensional indicator data of the analysis model. The feedback optimization module drives the dynamic rule generator to create or optimize corresponding detection rules based on the multi-dimensional indicator data.

8. The method according to claim 7, characterized in that, The method further includes: The continuous learning module uses a reinforcement learning mechanism to evaluate the long-term effects of governance and analysis strategies using a preset reward function, and dynamically adjusts system parameters based on the policy gradient method. The visualization module provides users with a multi-dimensional visualization dashboard, displaying real-time quality score curves, anomaly trend heatmaps, and governance execution success rates, and supports correlation analysis between data quality scores and business indicators; The security management module provides at least one of the following security supports for each module: unified authentication, access control isolation, operation log auditing, and data anonymization.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data analysis and governance method based on adaptive rule engine and model fusion as described in any one of claims 7-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information transmission implementation program, which, when executed by a processor, implements the steps of the data analysis and governance method based on adaptive rule engine and model fusion as described in any one of claims 7-8.

Images