A data quality intelligent repairing method based on dynamic rule evolution

Abstract

This invention discloses an intelligent data quality repair method based on dynamic rule evolution, belonging to the field of data quality management technology. It includes: constructing a knowledge graph based on the physical storage structure information and business logic of structured data; performing deep learning on the knowledge graph using a graph neural network, and dynamically generating a global topology view based on the learning results; combining historical damage patterns and the global topology view, using reinforcement learning to generate the optimal structured data scanning path, and identifying associated anomalies in abnormal partitions within the optimal path scanning results using a graph neural network; constructing a multimodal association subgraph based on the associated anomalies, repairing structural defects in the multimodal association subgraph using the correct physical structure deduced through the graph neural network, and scoring the reliability of the repair results to form a closed loop for structured data governance. This method can adapt to constantly evolving data patterns and new anomalies, continuously improving the system's robustness and autonomy in dealing with complex data problems, and reducing long-term operation and maintenance costs.

Description

Technical Field

[0001] This invention relates to the technical field of data quality management, and specifically to a data quality intelligent repair method based on dynamic rule evolution. Background Technology

[0002] With the advent of the big data era, power systems face the challenge of managing and maintaining massive amounts of structured data. The quality of data such as equipment ledgers, telemetry data, and marketing records directly impacts business decisions, system operational safety, and efficiency. Structured data contains numerous non-explicit, cross-system, multi-hop business logic relationships and complex hierarchical dependencies. Traditional methods, typically based on single data tables or simple associations, struggle to identify and model deep, implicit dependencies, leading to incomplete anomaly detection and difficulty in pinpointing the root cause. Databases are the core carriers of structured data storage, and the health of their underlying physical structures, such as B-tree indexes, data page organization, data block linked lists, and page validation, is crucial to data access performance and system stability. As a core structure for accelerating data queries, physical damage to B-tree indexes, such as internal pointer corruption, out-of-order leaf nodes, or corrupted root or intermediate pages, can cause a sharp decline in query performance, data inaccessibility, or even database crashes. However, the repair methods provided by existing data quality tools and database management systems are often limited to logical-level data correction or coarse-grained global reconstruction of the physical structure. Such global reconstruction is not only time-consuming and may require downtime, seriously affecting business continuity, but also lacks the ability to accurately locate and repair specific damage, thus failing to fundamentally solve the problem and improve the system's resilience.

[0003] Facing increasingly large and interconnected structured data networks, discovering and locating the root causes of multi-hop association anomalies and deep-seated physical corruption requires significant manpower and time. Traditional diagnostic processes are often linear and reactive, lacking a global perspective and intelligent path planning, making it difficult to quickly pinpoint the core of the problem. Most existing remediation technologies focus on correcting data content or employ highly invasive global reconstruction operations. Their ability to repair the underlying physical structure of the database, especially the fine-grained repair of B-tree indexes, is severely inadequate. Traditional root page repair may only attempt to restore the root page itself, but if the root page's anomaly stems from deep-seated problems in its child pages, this repair is merely a stopgap measure and may even introduce new data inconsistencies or corruption during the recovery process. Traditional data quality management systems are typically static, relying on pre-defined rules and unable to learn from historical anomalies and remediation experiences. Faced with constantly evolving data patterns and new anomalies, these systems lack the ability to self-evolve and continuously improve. In summary, existing technologies have significant shortcomings in the intelligent anomaly detection and accurate repair of structured data, especially its underlying physical structure. There is an urgent need for an automated system that can deeply understand the multi-level correlation of data, intelligently identify and repair deep physical damage, and has adaptive learning capabilities to ensure the data integrity, high availability, and long-term stability of critical power systems. Summary of the Invention

[0004] In view of the aforementioned problems, this invention is proposed. Therefore, this invention provides a data quality intelligent repair method based on dynamic rule evolution to address the shortcomings of traditional data quality management in understanding complex relationships, repairing underlying physical structures, anomaly detection efficiency, limitations in repair methods, and insufficient adaptive capabilities.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a data quality intelligent repair method based on dynamic rule evolution, comprising:

[0006] Obtain structured data from heterogeneous data sources in the power grid, and extract the physical storage structure information and business logic of the structured data;

[0007] The business logic is integrated with the physical storage structure information to construct a knowledge graph;

[0008] The knowledge graph is subjected to deep learning using a graph neural network, and a global topological view is dynamically generated based on the learning results of the graph neural network.

[0009] Combining historical damage patterns and a global topology view, reinforcement learning is used to generate an optimal structured data scanning path, and the graph neural network is used to identify the associated anomalies of abnormal partitions in the optimal path scanning results.

[0010] Based on the aforementioned association anomalies, a multimodal association subgraph is constructed. The correct physical structure is deduced through the graph neural network, the structural defects in the multimodal association subgraph are repaired, and the reliability of the repair results is scored, thus forming a closed loop of structured data governance.

[0011] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the extraction of the physical storage structure information and business logic of the structured data includes:

[0012] Extract multi-level metadata from heterogeneous data sources, obtain explicit metadata, and analyze unstructured documents through keyword extraction and relationship identification;

[0013] Analyze and identify patterns in business documents, historical data patterns, application code, and data streams to extract non-explicit business logic relationships between structured data;

[0014] The database system view is parsed, the underlying binary file format is analyzed, and the monitoring operation logs are monitored to extract the physical storage structure information of the database. The physical storage structure information includes the node pointers of the B-tree, the inter-page linked list, the checksum mode, and the physical address pointers.

[0015] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the method involves: fusing the business logic with physical storage structure information to construct a knowledge graph, including:

[0016] The extracted explicit metadata, non-explicit business logic relationships, data lineage information, and key database physical storage structure information are integrated to construct a knowledge graph;

[0017] The nodes of the knowledge graph include tables, fields, records, devices, metrics, physical pages, and index nodes, and the edges of the knowledge graph include logical associations, physical associations, data lineages, and inheritance relationships.

[0018] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the method includes: performing deep learning on the knowledge graph using a graph neural network, and dynamically generating a global topology view based on the learning results of the graph neural network, including:

[0019] Graph neural networks learn node features through two layers of graph convolutional layers; node features include metadata, business logic features, data statistical features, and underlying physical storage attributes.

[0020] Perform the first layer graph convolution operation to aggregate first-order neighbor information, apply the ReLU activation function to introduce non-linear characteristics, and apply the Dropout layer to randomly discard some node features to prevent overfitting.

[0021] Perform a second-layer graph convolution operation to aggregate higher-order neighbor information and learn the logical-physical relationship patterns between nodes;

[0022] Based on the global topology learned by the graph neural network, physical layer, logical layer, and application layer views are generated according to the target requirements.

[0023] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, it further includes: performing importance quantification scoring on nodes in the constructed global topology graph to identify the core nodes with the greatest impact on the system; the calculation of the importance quantification score is expressed as:

[0024] ;

[0025] in, For nodes Importance rating The damping coefficient is... Pointing to a node All nodes The set, For set Any node in, For nodes Importance rating For nodes The set of all outgoing chains, For nodes The degree of departure.

[0026] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the method includes: combining historical damage patterns and global topology view, using reinforcement learning to generate an optimal structured data scanning path, and identifying the associated anomalies of abnormal partitions in the optimal path scanning results based on the graph neural network, including: association analysis based on graph neural network, according to the abnormal partitions in the scanning results of the optimal path, giving an abnormal data probability in the abnormal partition, discovering anomalies in the data, and extracting abnormal data from the abnormal partitions;

[0027] By learning and reasoning about the graph structure, the edge weights are determined by the frequency of pointers between tables, the depth of dependencies, and the strength of associations. The graph neural network learns node features through multi-layer aggregation, infers the propagation path of anomalies in the graph, calculates the probability of anomalies in other data areas, and finally locates specific abnormal data.

[0028] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the graph neural network learns node features through multi-layer aggregation, infers the propagation path of anomalies in the graph, calculates the anomaly probability of other data regions, and finally locates specific anomalous data, including: aggregating neighbor features to generate node representations.

[0029] ;

[0030] in, For nodes In the Layer feature representation, It is a non-linear activation function. For the first The learnable weight matrix of the layer, It is the mean aggregation function. For nodes The set of all neighboring nodes, For nodes In the Layer feature representation, For nodes In the Layer feature representation;

[0031] The abnormal score is calculated as follows:

[0032] ;

[0033] in, For nodes The final feature representation after learning by the graph neural network The feature representation of cluster centers formed by normal data points. It is an L2 norm.

[0034] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the construction of a multimodal association subgraph based on the association anomaly includes: when the root page of the B-tree index is detected to be damaged, constructing a multimodal association subgraph based on the B-tree index damage;

[0035] Based on the existing index structure meta-information in the knowledge graph, and combined with the traversal of the database page directory, all leaf pages belonging to the B-tree index are located and filtered out. Each leaf page contains a complete and ordered range of key values, the physical address of the actual data row, and bidirectional physical pointers between leaf pages.

[0036] The leaf pages and the damaged index root pages are collectively abstracted into a multimodal association subgraph containing logical and physical relationships. The multimodal association subgraph is represented as follows:

[0037] ;

[0038] Among them, node set Including the root page node of the B-tree Intermediate page nodes Leaf page node and underlying data page nodes Each node Having feature vectors edge set This includes edges representing physical pointer links, doubly linked list edges between leaf pages, logical order relationships of records within a page, parent-child relationships between pages, and the binding relationship between checksums and page content.

[0039] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the repair of structural defects in the multimodal association subgraph through the correct physical structure derived by the graph neural network includes: the graph neural network aggregating the information of its neighbors on the leaf page nodes;

[0040] For each leaf page node Their neighbor information Calculated using the aggregation function AGG:

[0041] ;

[0042] node Its updated feature representation is as follows:

[0043] ;

[0044] in, For nodes Updated feature representation, and They are nodes and his neighbors The original characteristics, For nodes and his neighbors Features of the edges between them For nodes The set of neighboring nodes, It is an aggregate function. For splicing operations, The weight matrix is ​​a learnable matrix;

[0045] Using the aggregated leaf page feature set as input, a multi-layer graph neural network encoder is used to infer the correct structure of its parent node. The inference process is represented as follows:

[0046] ;

[0047] in, For the inferred root page or middle page High-level feature representation, For the graph neural network encoder model, It is the set of updated features of all leaf pages associated with the current parent node. It is the set of updated features of all intermediate pages associated with the current parent node;

[0048] Based on the statistical distribution and sorting of the key value range of the leaf pages, and combined with the balance principle of the B-tree, the optimal key value split point is predicted; based on the physical address of the leaf page or intermediate page and their logical position after sorting, the correct physical subpage address pointed to by the root page is predicted.

[0049] Based on the contextual information of surrounding healthy nodes and the learned patterns, the correct physical structure state of the damaged root page is corrected.

[0050] As a preferred embodiment of the intelligent data quality repair method based on dynamic rule evolution described in this invention, the reliability score of the repair result includes:

[0051] A preset confidence threshold is used to score the reliability of the repair solution generated by the graph neural network in multiple dimensions. If the confidence score is not less than the confidence threshold, the result is deemed reliable, the repair is automatically executed, and high-quality data is output. If the confidence score is less than the confidence threshold, the repair is not executed, and a manual review process is triggered.

[0052] Human experts combine their professional knowledge to make the final judgment and correction, and the results after human review and correction are stored and archived, forming a closed loop of structured data governance.

[0053] The beneficial effects of this invention are as follows: This invention identifies and locates anomalies in business logic associations and critical damage to the underlying physical structure of the database through graph neural networks. It can drive the generation of refined underlying database operation instructions, thereby achieving local and intelligent reconstruction of the B-tree index physical structure, avoiding the time-consuming and high-risk global reconstruction process. This invention realizes the intelligent and automated evolution of high-quality datasets, enabling them to adapt to constantly evolving data patterns and new anomalies, continuously improving the system's robustness and autonomy in dealing with complex data problems, and reducing long-term operation and maintenance costs. By proactively adjusting the fragmentation rate of the B-tree index, maintaining the balance factor, and scoring the importance of nodes after repair, the index structure is further optimized, thereby comprehensively improving query efficiency and overall system stability. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of a data quality intelligent repair method based on dynamic rule evolution, provided as an embodiment of the present invention. Detailed Implementation

[0056] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0057] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a data quality intelligent repair method based on dynamic rule evolution, including:

[0058] S100: Obtain structured data from heterogeneous data sources in the power grid, and extract the physical storage structure information and business logic of the structured data;

[0059] Furthermore, extracting the physical storage structure information and business logic of structured data includes:

[0060] Extract multi-level metadata from heterogeneous data sources, obtain explicit metadata, and analyze unstructured documents through keyword extraction and relationship identification;

[0061] Analyze and identify patterns in business documents, historical data patterns, application code, and data streams to extract non-explicit business logic relationships between structured data;

[0062] The database system view is parsed, the underlying binary file format is analyzed, and the monitoring operation logs are monitored to extract the physical storage structure information of the database. The physical storage structure information includes the node pointers of the B-tree, the inter-page linked list, the checksum mode, and the physical address pointers.

[0063] In step S100 of this embodiment of the invention, raw structured data is obtained from various heterogeneous data sources in the power grid industry, such as enterprise resource planning equipment ledgers, marketing system customer files, various system logs, and configuration information. The data is further standardized and unified, including aspects such as type, encoding, time format, and basic cleaning, such as handling missing and outlier values, and supports distributed, efficient processing.

[0064] It should be noted that the structured data extraction, conversion, and loading methods in step S100 of the embodiments of the present invention are conventional technical operations in the art, and the present invention does not specifically limit them.

[0065] S200: Integrates business logic with physical storage structure information to construct a knowledge graph;

[0066] Furthermore, by integrating business logic with physical storage structure information to construct a knowledge graph, the following steps are taken:

[0067] The extracted explicit metadata, non-explicit business logic relationships, data lineage information, and key database physical storage structure information are integrated to construct a knowledge graph;

[0068] The nodes of a knowledge graph include tables, fields, records, devices, metrics, physical pages, and index nodes. The edges of a knowledge graph include logical associations, physical associations, data lineage, and inheritance relationships.

[0069] In step S200 of this embodiment of the invention, advanced data parsing techniques are employed, such as pattern recognition-based syntax parsers, rule engines, data dictionary analysis, database system view queries, database internal log analysis, and keyword extraction and relationship recognition of unstructured business documents, to intelligently and automatically extract multi-level metadata. Through in-depth analysis and pattern recognition of business documents, historical data patterns, application code, and data flows, implicit business logic relationships between structured data are extracted. For example, even if the database has no foreign key constraints, the implicit business relationship between the sensor data of a transformer and the transformer's operational status in the equipment ledger can be identified.

[0070] In step S200 of this embodiment of the invention, by parsing the database system view, such as obtaining the physical storage information and index structure of the table, analyzing the underlying binary file format of the database, such as parsing the B-tree index page structure and the internal structure of the data page, and monitoring the database operation log, the underlying physical storage structure information of the database is accurately identified and extracted, such as the internal node pointers of the B-tree index, the leaf node structure, the linked list relationship between data blocks, the checksum mode of the data page, and the pointing relationship of the physical address, etc.

[0071] In step S200 of this embodiment of the invention, the extracted explicit metadata, implicit business logic relationships, data lineage information, and key database physical storage structure information are integrated to construct a unified, multi-dimensional knowledge graph. This knowledge graph contains different types of nodes, such as tables, fields, records, devices, metrics, physical pages, and index nodes, and different types of edges, such as logical associations, physical associations, data lineages, and inheritance relationships. A graph database, such as Neo4j or Nebula Graph Database, is used for storage, providing rich graph structure inputs for graph neural networks.

[0072] Furthermore, the knowledge graph is represented as:

[0073] ;

[0074] in, For the constructed knowledge graph, The set of all nodes in the knowledge graph. It is the set of all edges in the knowledge graph.

[0075] It should be noted that this embodiment of the invention revolutionarily extends structured data from the business logic layer and data lineage layer to the underlying physical structure layer of the database through the construction of a multimodal knowledge graph driven by a graph neural network. The system uses advanced data parsing techniques, such as in-depth analysis of database system views, internal logs, and physical file formats, to accurately extract and fuse physical layer metadata, constructing a unified global knowledge graph that includes logical and physical relationships. This enables the graph neural network to learn and represent the true full picture of structured data, laying the foundation for subsequent deep anomaly detection and precise repair.

[0076] Furthermore, by deeply fusing multi-source and multi-level metadata through graph neural networks, a multimodal global topological view of structured data is constructed, enabling intelligent repair of underlying physical structures such as B-tree indexes. The underlying physical storage units of the database, such as data blocks, data pages, B-tree index nodes, and physical addresses, are further abstracted as low-level nodes in the graph. Edges not only represent traditional business logic relationships, such as the inclusion relationship, data lineage, and parent-child relationship between the Device table and the Site table, but more importantly, graph neural networks can establish physical-level relationships based on the parsed physical layer metadata. For example:

[0077] Index link edges directly represent the physical pointer relationships between nodes within a B-tree index (non-leaf nodes pointing to child nodes, leaf nodes pointing to data rows), as well as the mapping relationship between index pages and data pages.

[0078] Data page linked list edges represent the relationship between adjacent data blocks in the same data table that may be connected by linked list pointers in physical storage. Checksum association edges represent the logical and physical binding relationship between a data page and its checksum field. This graph structure abstraction spanning the logical layer, data content layer, and physical storage layer enables graph neural networks to learn the true full picture of structured data, including its most subtle internal physical structural features. This provides a unique global view for subsequently discovering deep, hidden structured data anomalies, especially damage to the underlying physical structure of the database.

[0079] Furthermore, by using graph neural networks, such as graph data inductive learning to learn node features, and aggregating neighborhood information, such as the association features between the Device information table and the Site information table, the topology structure can be optimized.

[0080] S300: Utilizes graph neural networks to perform deep learning on knowledge graphs and dynamically generates a global topology view based on the learning results of graph neural networks;

[0081] Furthermore, deep learning of knowledge graphs is performed using graph neural networks, and a global topological view is dynamically generated based on the learning results of the graph neural network, including:

[0082] Graph neural networks learn node features through two layers of graph convolutional layers; node features include metadata, business logic features, data statistical features, and underlying physical storage attributes.

[0083] Perform the first layer graph convolution operation to aggregate first-order neighbor information, apply the ReLU activation function to introduce non-linear characteristics, and apply the Dropout layer to randomly discard some node features to prevent overfitting.

[0084] Perform a second-layer graph convolution operation to aggregate higher-order neighbor information and learn the logical-physical relationship patterns between nodes;

[0085] Based on the global topology learned by the graph neural network, physical layer, logical layer, and application layer views are generated according to the target requirements.

[0086] For example, graph neural networks learn complex logical-physical relationship patterns between nodes by aggregating neighborhood information. For instance, a graph neural network can simultaneously aggregate features of the Device table and its logically related Site table, as well as features of the B-tree index pages on which the Device table physically depends, thereby constructing node embeddings that can simultaneously reflect both logical and physical states.

[0087] It should be noted that graph neural networks can automatically learn and extract high-level features from massive heterogeneous, multi-layered structured data, revealing deep dependencies and structural patterns that are difficult to capture by traditional methods, significantly improving the ability to understand data complexity, and providing high-dimensional, semantically rich feature representations for subsequent anomaly detection and repair.

[0088] In step S300 of this embodiment, based on the global topology learned by the graph neural network, this invention can dynamically generate and present physical layers, such as storage links and B-tree index structures, logical layers, such as business relationships and data lineages, and application layers, such as customized topology views of data dependencies required for specific analysis scenarios. It supports multi-granularity drill-down and anomaly propagation visualization: users can drill down layer by layer according to power grid business levels such as site → equipment → defect, intuitively displaying the propagation path and impact range of anomalies at different levels. For example, how an anomaly in telemetry data of a data acquisition and monitoring control system affects decision reports through logical lineage, and how damage to the corresponding B-tree index at the database level affects data query performance.

[0089] Furthermore, by employing advanced layout algorithms such as force-directed algorithms, node overlap is intelligently reduced, and the visual display of complex business relationships is optimized. This makes the generated topology map more concise and aesthetically pleasing, significantly reducing the cognitive burden on database administrators and business experts in understanding complex data relationships, and improving the efficiency of anomaly location and analysis.

[0090] Furthermore, it also includes: quantifying the importance of nodes in the constructed global topology graph to identify the core nodes that have the greatest impact on the system;

[0091] The importance quantification score is calculated as follows:

[0092] ;

[0093] in, For nodes Importance rating The damping coefficient is... Pointing to a node All nodes The set, For set Any node in, For nodes Importance rating For nodes The set of all outgoing chains, For nodes The degree of departure.

[0094] Furthermore, the damping coefficient is a constant between 0 and 1, typically with a value of 0.85, representing the probability that a user will randomly jump to any node.

[0095] It should be noted that this invention uses PageRank-based node importance scoring and adaptive optimization to prioritize anomaly detection resource allocation for high-importance nodes, such as equipment operation status tables and key real-time telemetry data tables in the power grid, and frequently used nodes, such as equipment function location fields. This ensures that data with the greatest impact on critical business operations receives the highest priority for inspection. When adding a node, the system can quickly determine its necessity for association with core nodes through scoring and automatically adjust the topology layout to maintain the stability of the core structure, ensuring that the system remains efficient and accurate even with dynamic data changes. When simulating the impact of critical node failures, such as damage to core equipment ledger tables or their indexes, on the overall topology, the cascading effect is assessed through importance scoring, thereby optimizing the topology structure or designing contingency plans. This significantly improves the system's fault tolerance and resilience, ensuring the continuity of power grid operations. The system can intelligently focus on the most critical and vulnerable parts of structured data, achieving intelligent resource allocation and risk early warning management, fundamentally improving the intelligence and robustness of data governance.

[0096] Taking a power grid business scenario as an example, in the power grid topology diagram, the node Site_id=04 represents a critical substation, and the node Device_ID=0103TR001 represents a main transformer within that substation, connected by an edge. This edge is learned by the graph neural network and assigned a weight of 0.8, indicating a high-frequency and important association. Simultaneously, the graph neural network also identifies and marks the correlation threshold between the transformer's current curve and the current curves of other equipment in the same area. This information collectively reflects the importance of the association and related data, enabling the system to prioritize checking the transformer's current data and its supporting physical index during subsequent detection. Furthermore, when anomalies are detected, the system can combine this important information to generate a more prioritized and accurate repair plan.

[0097] It should be noted that when dealing with large volumes of structured data, a full scan is time-consuming and resource-intensive; while a partial scan may miss critical anomalies that propagate across tables. By combining intelligent path planning and deep analysis using graph neural networks, the blind spots and limitations of traditional methods are overcome.

[0098] Furthermore, by leveraging historical high-frequency corruption patterns and statistically learning from historical structured data anomalies, the system identifies which data tables, partitions, or physical data blocks were more prone to corruption or inconsistency in the past, guiding the scanning process. Combined with a constructed global topology view of the structured data, the system intelligently generates the optimal structured data scanning path using reinforcement learning models or greedy algorithms. This path planning mechanism prioritizes covering data areas, tables, or partitions that are historically identified as high-risk or highly corrupted. Compared to traditional linear scanning or random sampling, this intelligent path planning mechanism is more efficient and targeted, maximizing coverage of all potentially corrupted structured data areas with minimal resource consumption. It is particularly suitable for large-scale, high-concurrency database environments, significantly improving detection efficiency.

[0099] S400: Combining historical damage patterns and global topology views, it uses reinforcement learning to generate the optimal structured data scanning path and uses graph neural networks to identify the associated anomalies of abnormal partitions in the optimal path scanning results.

[0100] Furthermore, by combining historical damage patterns and global topology views, reinforcement learning is used to generate optimal structured data scanning paths. Based on graph neural networks, the correlation anomalies of abnormal partitions in the optimal path scanning results are identified, including: correlation analysis based on graph neural networks, based on abnormal partitions in the optimal path scanning results, an abnormal data probability is given in the abnormal partitions, anomalies in the data are found, and abnormal data is extracted from the abnormal partitions.

[0101] By learning and reasoning about the graph structure, the edge weights are determined by the frequency of pointers between tables, the depth of dependencies, and the strength of associations. The graph neural network learns node features through multi-layer aggregation, infers the propagation path of anomalies in the graph, calculates the probability of anomalies in other data areas, and finally locates specific abnormal data.

[0102] In step S200 of this embodiment, the graph neural network-based association analysis identifies anomalous data probabilities within the abnormal partitions in the optimal path's scan results. This process detects anomalies in the data and pinpoints specific anomalous data within these partitions. Through graph structure learning and reasoning, edge weights are determined by the frequency of pointers between tables, dependency depth, and association strength. The graph neural network, through multi-layer aggregation learning of node features, can identify abnormal links between tables, such as broken index pointers and inconsistent parent-child page checksums. This graph neural network-based analysis method enables the model to maintain high diagnostic accuracy even in the face of distributed corruption or cascading failures.

[0103] Traditional anomaly detection methods mostly only scan a single region, making it difficult to perform cross-regional anomaly scanning. Even those few methods that can perform cross-regional anomaly scanning still suffer from low efficiency, low accuracy, and high resource consumption. This paper addresses this issue by using graph structure learning and reasoning. By analyzing tables, partitions, and relationships, and then based on the scanned known anomaly regions, the paper constructs a graph structure. Entities in the database are abstracted as graph nodes, and relationships such as foreign key constraints, index structures, and field constraints are abstracted as graph edges. These three factors are then transformed into computable numerical features, and their weighted sum is used to determine the final edge weights. The specific process is as follows:

[0104] ① The frequency of pointers between tables is the activity of references between tables, which can be quantified as the number of times the pointer from Marketing Archives in Table A to User Information in Table B is accessed within a statistical period; for example, if the User Information ID in Marketing Archives table points to the User Information table, and this pointer is queried 100 times in 1 hour, then the pointer frequency of the A and B sides can be quantified as 100 times / hour.

[0105] ② Dependency depth refers to the relationship between tables, which can be quantified by the length of the dependency chain. If table A (marketing ledger) directly depends on table B (user information), the dependency depth from A to B is 1. If table A (user electricity consumption) depends on table B (marketing ledger), and B depends on table C (user information), the dependency depth from A to B to C is 2, and so on. For example, if the user electricity consumption table depends on the user information table, which in turn depends on the marketing ledger table, the dependency depth from A to B is 2, and the dependency depth from A to B to C is 3.

[0106] ③ The strength of the association is determined by the frequency of data interaction between tables and the tightness of business logic, and needs to be divided into two levels: business level and data level.

[0107] 1) Business level: If two tables appear at the same business level, the relationship is strong;

[0108] 2) Data level: If data in two tables is queried simultaneously and the amount of data synchronized between the tables is high, then the correlation is high.

[0109] For example, if there are 50,000 joint queries between the marketing ledger and the user information table per day, and 30,000 user information changes are made to the marketing ledger, then the strength of the association between the two can be quantified as (50,000 + 3,000) / day = 53,000 times / day.

[0110] ④ The quantified values ​​of the three factors are denoted as: pointer frequency, dependency depth, and association strength are converted into edge weights;

[0111] ⑤ Initial fusion: Assume that the edge weights are a linear relationship between the three factors.

[0112] ;

[0113] in, , , The coefficients to be learned reflect the importance of each factor. For pointer frequency, For depth-dependent purposes, For correlation strength, This is a transformation function that depends on depth. Since the effect may weaken as the depth increases, it is usually taken as the reciprocal or exponentially decaying, such as... and To avoid excessive depth and excessively high weights.

[0114] For example, if table A directly depends on table B, the dependency depth is... ,but The impact is strongest if table A depends on table C through an intermediate table; the dependency depth is... ,but The impact is reduced; if the dependency chain is longer and deeper... ,but The impact has further weakened.

[0115] ⑥ Perform graph structure reasoning and verification by optimizing feedback coefficients:

[0116] 1) Infer whether the target of the inference task is abnormal. For example, if a node in the graph is in an abnormal state, predict that its neighboring nodes are abnormal.

[0117] 2) Add a feedback mechanism: if the inference result of the current edge weight deviates significantly from the actual data, adjust the loss function, such as cross-entropy loss. , , The value makes the edge weights more reflective of the relation weights.

[0118] The cross-entropy loss function is expressed as:

[0119] ;

[0120] Among them, real labels A value of 1 indicates that abnormal propagation has actually occurred; this is the true label. A value of 0 indicates that the event did not occur; the model predicts the probability. The probability of abnormal propagation of the edge, calculated based on the current edge weight, ranges from 0 to 1.

[0121] ⑦ Through multiple iterations, backpropagation is used to calculate L pairs. , , The gradient of the value, adjusting the parameters to reduce L to a lower level, and the more reasonable the subsequent weight settings:

[0122] Furthermore, the graph neural network integrates the features of a node with those of its neighboring nodes through a core multi-layer aggregation pattern, thereby identifying abnormal cross-table links:

[0123] The first layer of aggregation: nodes merge the features of their directly connected neighbors. For example, if the marketing ledger is directly associated with the user information table, then the marketing table merges the features of the user information table, and vice versa, capturing anomalies caused by direct dependencies.

[0124] Node features = Self-features + (Neighbor features × Edge weights)

[0125] The second layer of aggregation: Nodes merge the features of their neighbors. For example, if a user's electricity consumption meter is associated with a marketing table, and the marketing table is associated with a user information table, then the user's electricity consumption meter merges the first layer features of the marketing table, and the first layer features of the marketing table already include the features of the user information table, thus capturing the abnormal cascading caused by indirect dependencies.

[0126] Node features = first-layer features + (neighbors' first-layer features × edge weights)

[0127] Multi-layer aggregation, as the number of aggregation layers increases, node features will be integrated with the features of more distant neighbors (third-order and fourth-order neighbors), thereby identifying distributed abnormal link data that are not directly related but have implicit dependencies. Graph neural networks can then identify common links that are distributedly damaged.

[0128] Furthermore, graph neural networks learn node features through multi-layer aggregation, infer the propagation path of anomalies in the graph, calculate the probability of anomalies in other data regions, and ultimately locate specific anomalous data, including:

[0129] Aggregate neighbor features to generate node representations:

[0130] ;

[0131] in, For nodes In the Layer feature representation, It is a non-linear activation function. For the first The learnable weight matrix of the layer, It is the mean aggregation function. For nodes The set of all neighboring nodes, For nodes In the Layer feature representation, For nodes In the Layer feature representation;

[0132] The abnormal score is calculated as follows:

[0133] ;

[0134] in, For nodes The final feature representation after learning by the graph neural network The feature representation of cluster centers formed by normal data points. It is an L2 norm.

[0135] It should be noted that this invention utilizes the powerful graph learning and reasoning capabilities of graph neural networks to perform anomaly pattern recognition on the constructed global topology view. Graph neural networks can accurately identify multi-hop business association anomalies at the logical level, as well as key underlying physical structures of the database, especially B-tree index corruption patterns, by analyzing the feature propagation and structural patterns of data on the graph. Simultaneously, the introduction of reinforcement learning for intelligent planning of diagnostic paths significantly improves the efficiency and accuracy of anomaly localization.

[0136] It should be noted that traditional B-tree index corruption repair methods, especially those targeting the core index root page (INDEX_ROOT), typically only attempt to recover the damaged root page itself. This not only carries the risk of data loss or further corruption—for example, reliance on outdated backups leading to irreparable damage, or the root page anomaly actually stemming from a deep anomaly in its child pages—but also fail to address the root cause. Furthermore, the repair process is time-consuming and requires downtime, severely impacting business continuity. This invention, by introducing the powerful global association analysis and structural reasoning capabilities of graph neural networks, achieves intelligent reconstruction of the physical structure of the B-tree index, fundamentally improving the accuracy, success rate, and system resilience of the repair process.

[0137] Specifically, when the system detects that the INDEX_ROOT page of a certain B-tree index is corrupted, the present invention no longer stops at isolated repair attempts on the root page itself, but initiates an intelligent repair process.

[0138] S500: Constructs a multimodal association subgraph based on association anomalies, inversely derives the correct physical structure through graph neural networks, repairs structural defects in the multimodal association subgraph, and scores the credibility of the repair results to form a closed loop of structured data governance.

[0139] Furthermore, constructing a multimodal association subgraph based on association anomalies includes: when a B-tree index root page is detected to be corrupted, constructing a multimodal association subgraph based on the B-tree index corruption;

[0140] Based on the existing index structure meta-information in the knowledge graph, and combined with the traversal of the database page directory, all leaf pages belonging to the B-tree index are located and filtered out. Each leaf page contains a complete and ordered range of key values, the physical address of the actual data row, and bidirectional physical pointers between leaf pages.

[0141] The leaf pages and the damaged index root pages are collectively abstracted into a multimodal association subgraph containing logical and physical relationships. The multimodal association subgraph is represented as follows:

[0142] ;

[0143] Among them, node set Including the root page node of the B-tree Intermediate page nodes Leaf page node and underlying data page nodes Each node Having feature vectors edge set This includes edges representing physical pointer links, doubly linked list edges between leaf pages, logical order relationships of records within a page, parent-child relationships between pages, and the binding relationship between checksums and page content.

[0144] Specifically, the feature vector contains information such as page type, key value range, page physical address, and checksum.

[0145] It should be noted that when a corrupted root page of a B-tree index is detected, this invention is no longer limited to the root page itself. Instead, it leverages the powerful capabilities of graph neural networks to intelligently track and associate all leaf nodes and intermediate pages pointed to by the root page, forming a complete subgraph containing logical and physical relationships. Through joint diagnosis and structural reasoning of this subgraph, even if key information in the root page is lost or incorrect, the graph neural network can intelligently deduce the correct physical structure state of the root page, such as the correct key value range and child node pointers, based on its deep understanding of the normal B-tree structure pattern and its ability to reverse-engineer the true information of the leaf pages. The graph neural network drives the generation of a series of refined low-level database operation instructions to accurately correct broken pointers and erroneous page references, thereby achieving local and intelligent reconstruction of the physical structure of the B-tree index. This significantly improves the accuracy and success rate of repair, avoiding the huge performance overhead and business downtime risks associated with global reconstruction.

[0146] It should be noted that this invention utilizes a trained graph neural network to perform joint diagnosis and structural reasoning on multimodal relational subgraphs. Even if some key information on the INDEX_ROOT page is lost or incorrect, the graph neural network can still perform joint diagnosis and structural reasoning on the subgraph. It uses deep learning and reasoning to reconstruct the correct structure.

[0147] Furthermore, the correct physical structure derived through graph neural networks can be used to repair structural defects in multimodal association subgraphs, including: graph neural networks aggregating information about the neighbors of leaf page nodes;

[0148] For each leaf page node Their neighbor information Calculated using the aggregation function AGG:

[0149] ;

[0150] node Its updated feature representation is as follows:

[0151] ;

[0152] in, For nodes Updated feature representation, and They are nodes and his neighbors The original characteristics, For nodes and his neighbors Features of the edges between them For nodes The set of neighboring nodes, It is an aggregate function. For splicing operations, The weight matrix is ​​a learnable matrix;

[0153] Using the aggregated leaf page feature set as input, a multi-layer graph neural network encoder is used to infer the correct structure of its parent node. The inference process is represented as follows:

[0154] ;

[0155] in, For the inferred root page or middle page High-level feature representation, For the graph neural network encoder model, It is the set of updated features of all leaf pages associated with the current parent node. It is the set of updated features of all intermediate pages associated with the current parent node;

[0156] Based on the statistical distribution and sorting of the key value range of the leaf pages, and combined with the balance principle of the B-tree, the optimal key value split point is predicted; based on the physical address of the leaf page or intermediate page and their logical position after sorting, the correct physical subpage address pointed to by the root page is predicted.

[0157] Based on the contextual information of surrounding healthy nodes and the learned patterns, the correct physical structure state of the damaged root page is corrected.

[0158] It should be noted that graph neural networks, by learning from the normal B-tree structure pattern, can identify structural defects in subgraphs, such as broken pointers, incorrect key value ranges, and inconsistent checksums. Based on the contextual information of surrounding healthy nodes and the learned patterns, they can intelligently complete or correct the correct physical structure of the damaged root page. This deep reasoning capability of graph neural networks based on global subgraphs surpasses traditional repair methods that only focus on a single damaged page. It can start from the true and complete information of the underlying leaf pages, reverse-engineer and reconstruct the correct structure of the root page, and even identify the underlying causes of root page damage, such as an error in an intermediate page causing abnormal root page pointers. This assists the system in generating more intelligent and robust repair decisions, achieving intelligent reconstruction of the underlying physical structure.

[0159] In step S500 of this embodiment of the invention, the system utilizes the root page structure information derived from a graph neural network to accurately reconstruct or correct the INDEX_ROOT page. This typically involves sending specific instructions to the database management system's API or underlying interface to precisely write the calculated correct key value range and child node pointers. After repair, the system performs a rigorous verification process to ensure the thoroughness of the repair and data integrity.

[0160] This invention introduces a structural consistency metric to calculate the consistency deviation between the repaired root page structure and the actual leaf page information. The structural consistency metric is expressed as:

[0161] ;

[0162] in, The number of key value points. These are the key-value split points predicted by the graph neural network. These are key-value points calculated based on the actual data of the leaf pages.

[0163] It should be noted that the structural consistency metric measures the similarity between the predicted structure and the actual structure, with a value range of [0,1]. If the structural consistency metric is <0.9, it indicates that there is a significant deviation between the predicted structure and the actual structure. The system will prioritize marking the relevant leaf nodes as potentially having data anomalies and trigger the repair or in-depth diagnosis of the leaf pages.

[0164] Furthermore, by executing a key query that includes the index, the system verifies whether the repaired B-tree index can correctly and efficiently locate data, ensuring its functional recovery. After the repair is complete, the system automatically evaluates the index fragmentation rate, which quantifies the utilization of index pages. The calculation formula is as follows:

[0165] ;

[0166] When the fragmentation rate exceeds the preset fragmentation rate threshold, the system will intelligently trigger an index reorganization operation. By adjusting the page fill factor, it can effectively reduce space waste and I / O overhead, and further improve query efficiency.

[0167] In this embodiment of the invention, the fragmentation rate threshold can be set to 30%.

[0168] The system continuously monitors the health of the B-tree index. Key metrics include node fill rate and tree depth. The formula for calculating the node fill rate is:

[0169] ;

[0170] The system monitors the average node fill rate of the index. When the average fill rate falls below a preset threshold, it indicates excessive fragmentation within the index, leading to decreased storage efficiency and scan performance. Simultaneously, the system monitors the overall depth of the B-tree; unexpected increases in depth also suggest potential performance degradation. When these health metrics exceed reasonable ranges, the system triggers index maintenance operations, such as index rebuilding or reorganization, to optimize the node fill rate, reduce tree depth, and thus maintain the optimal B-tree structure and query performance.

[0171] In this embodiment of the invention, the fill rate threshold can be set to 70%.

[0172] It should be noted that this proactive structural optimization, combined with PageRank's node importance scoring, can quickly determine the necessity of adding or deleting data in relation to core nodes through scoring, automatically adjust the layout to maintain the stability of the core structure, improve the system's fault tolerance and resilience, and reduce the probability of similar physical structure problems recurring.

[0173] Taking the power grid business scenario as an example, in the power grid equipment asset management system, Device_ID is the core index. If its B-tree index INDEX_ROOT page is corrupted due to disk failure or abnormal write, making it impossible to quickly retrieve equipment information, it will immediately start:

[0174] Even if the root page is corrupted, the system can still quickly identify all leaf pages containing Device_ID through database metadata and extract their key value range (such as 0001 to 9999) and physical address.

[0175] Based on the complete information of these ordered leaf pages, the graph neural network intelligently calculates the key-value split point that the root page should have, such as pointing to the left subtree if Device_ID is less than 5000, pointing to the right subtree if Device_ID is greater than or equal to 5000, and its correct subpage pointer.

[0176] The system uses this correctly inferred information to directly correct the binary structure of the root page, rather than performing a full index reconstruction. For example, for an index entry with Device_ID A123, the graph neural network infers that it should be in the 5th index entry of the root page, pointing to the intermediate page with physical address 0xABCDEF, and the system directly writes this correction instruction.

[0177] After the repair, the system will check the fragmentation rate and health status of the Device_ID index and perform necessary reorganization and optimization to ensure that future queries of device information are always efficient and stable, thereby guaranteeing the rapid response and decision-making capabilities of power grid operation and maintenance.

[0178] Furthermore, the credibility of the repair results is scored, including:

[0179] A preset confidence threshold is used to score the reliability of the repair solution generated by the graph neural network in multiple dimensions. If the confidence score is not less than the confidence threshold, the result is deemed reliable, the repair is automatically executed, and high-quality data is output. If the confidence score is less than the confidence threshold, the repair is not executed, and a manual review process is triggered.

[0180] Human experts combine their professional knowledge to make the final judgment and correction, and the results of human review and correction are stored and archived to form a structured data governance closed loop.

[0181] In step S500 of this embodiment of the invention, a human-machine collaborative feedback mechanism is established to ensure the quality and reliability of the repair results, thereby forming a sustainable, reliable, and structured data governance closed loop. The accuracy and reliability of the repair results generated by the graph neural network model may be uncertain. The graph neural network is used to comprehensively evaluate the consistency, logical rationality, and physical integrity of the repaired data within the graph structure, assigning a reasonable confidence score to each recommended repair result.

[0182] The confidence score (0-100 points) is calculated based on the confidence level of the graph neural network prediction, such as the anomaly probability > 0.9 and the rule matching degree, if it conforms to the coding specifications.

[0183] The formula for calculating data credibility is:

[0184] ;

[0185] Among them, data credibility represents the final calculated overall data credibility score, which usually ranges from 0 to 1. The higher the score, the more reliable the data quality. For the first The normalized error rate of each data quality dimension, with values ​​ranging from 0 to 1; For the first The weight coefficients for each data quality dimension range from 0 to 1.

[0186] It should be noted that the data quality dimension These can represent missing rate, conflict rate, and delay rate, respectively. It could be the missing data rate. It can be the data conflict rate, etc., where the missing rate weight is 0.5, the conflict rate is 0.3, the latency rate is 0.2, and a result ≥0.9 is considered reliable.

[0187] Formula for calculating the credibility of a graph neural network model based on test set performance evaluation:

[0188] Model confidence = 0.6·AUC-ROC + 0.4·(1−prediction bias);

[0189] AUC-ROC is a model performance evaluation metric, with a value ranging from 0 to 1.

[0190] For example, if the AUC-ROC of any graph neural network model is 0.92 and the prediction bias is 0.05, the calculated confidence level is 0.87. If the confidence level reaches a certain score, automatic data repair is triggered, and high-quality data is output after repair. If the confidence level cannot reach the threshold, manual review is triggered, and the repaired value, confidence level, and evaluation basis are stored in the repair result table.

[0191] Furthermore, the system makes intelligent decisions based on the calculated confidence score. If the confidence score of the repair result meets a preset threshold, for example, higher than 90-95 points, the system will determine its high reliability and directly perform automated repair of the abnormal structured data, ultimately completing the abnormal data repair. This automated decision-making significantly improves repair efficiency.

[0192] If the confidence level of the repair result does not meet the preset threshold, or if it involves high-risk critical data, the original value and the graph neural network model's prediction value will be pushed to human review. Human reviewers will make the final judgment and correction based on their extensive professional knowledge and experience. The structured data that has been manually reviewed and corrected will be precisely labeled with anomaly correction tags and automatically incorporated into a trustworthy, traceable, and auditable structured data governance closed loop. High-quality corrected data after human review is automatically collected and used as retraining data for graph neural networks and reinforcement learning models. Graph neural networks will learn new anomaly patterns, repair strategies, and structural patterns from manually corrected cases. For example, if a human corrects a graph neural network's repair scheme for a specific type of physical damage, such as a specific type of damage to a B-tree index, these corrected structures will serve as new training samples, enabling the graph neural network to generate more accurate and robust solutions when encountering similar situations in the future. Reinforcement learning models will also adjust their reward function and state-action policy based on human feedback, optimizing the path selection for diagnosis and repair.

[0193] Through this continuous feedback and retraining mechanism, the present invention can continuously learn new anomaly patterns, optimize diagnostic strategies, and improve repair solutions, especially the accuracy and automation of B-tree index repair, forming a self-evolving and continuously improving intelligent data governance closed loop, thereby ensuring the long-term stability and efficiency of the system.

[0194] Taking the power grid business scenario as an example, the system uses a graph neural network to verify whether the repaired power grid equipment ledger data logically matches the actual assets and historical operating data. For example, it verifies whether the repaired equipment models conform to the equipment specifications of the substation and ensures the physical integrity of its database indexes and data pages. All repair and manual intervention records are recorded by the system and fed back into the graph neural network model, enabling it to identify and intelligently repair similar physical structure anomalies in power data that may occur in the future, such as damaged B-tree indexes in the topological association of the power grid geographic information system, or physical problems in a time-series index in real-time telemetry data storage.

[0195] It should be noted that this invention constructs a graph neural network-enhanced credibility assessment and data governance closed loop. After the repair is completed, the graph neural network performs a structural consistency assessment on the repair results. High-quality feedback data from manual review and correction is automatically collected and used as retraining samples for the graph neural network and reinforcement learning model. Through this continuous feedback and retraining mechanism, the system can continuously learn new anomaly patterns, optimize diagnostic strategies, and improve the accuracy of repair solutions, forming a self-evolving and continuously improving intelligent data governance closed loop, thereby ensuring the long-term stability and efficiency of the system.

[0196] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data quality intelligent repairing method based on dynamic rule evolution, characterized in that, include: Obtain structured data from heterogeneous data sources in the power grid, and extract the physical storage structure information and business logic of the structured data; The business logic is integrated with the physical storage structure information to construct a knowledge graph; The knowledge graph is subjected to deep learning using a graph neural network, and a global topological view is dynamically generated based on the learning results of the graph neural network. Combining historical damage patterns and global topology views, reinforcement learning is used to generate optimal structured data scanning paths. Based on the graph neural network, the association anomalies of abnormal partitions in the optimal path scanning results are identified. Specifically, this includes: association analysis based on graph neural networks, based on the abnormal partitions in the scanning results of the optimal path, giving an abnormal data probability in the abnormal partitions, discovering anomalies in the data, and extracting abnormal data from the abnormal partitions. Through graph structure learning and reasoning, it is determined that the edge weights are determined by the frequency of pointers between tables, the depth of dependency, and the strength of association. The graph neural network learns node features through multi-layer aggregation, infers the propagation path of anomalies in the graph, calculates the probability of anomalies in other data areas, and finally locates specific abnormal data. The graph neural network learns node features through multi-layer aggregation, infers the propagation path of anomalies in the graph, calculates the probability of anomalies in other data regions, and ultimately locates specific anomalous data, including: aggregating neighbor features to generate node representations. ； in, For nodes In the Layer feature representation, It is a non-linear activation function. For the first The learnable weight matrix of the layer, It is the mean aggregation function. For nodes The set of all neighboring nodes, For nodes In the Layer feature representation, For nodes In the Layer feature representation; The abnormal score is calculated as follows: ； in, For nodes The final feature representation after learning by the graph neural network The feature representation of cluster centers formed by normal data points. It is an L2 norm; Based on the aforementioned association anomalies, a multimodal association subgraph is constructed. The correct physical structure is deduced through the graph neural network, the structural defects in the multimodal association subgraph are repaired, and the reliability of the repair results is scored, thus forming a closed loop of structured data governance.

2. The intelligent data quality repair method based on dynamic rule evolution as described in claim 1, characterized in that, Extracting the physical storage structure information and business logic of the structured data includes: Extract multi-level metadata from heterogeneous data sources, obtain explicit metadata, and analyze unstructured documents through keyword extraction and relationship identification; Analyze and identify patterns in business documents, historical data patterns, application code, and data streams to extract non-explicit business logic relationships between structured data; The database system view is parsed, the underlying binary file format is analyzed, and the monitoring operation logs are monitored to extract the physical storage structure information of the database. The physical storage structure information includes the node pointers of the B-tree, the inter-page linked list, the checksum mode, and the physical address pointers.

3. The intelligent data quality repair method based on dynamic rule evolution as described in claim 2, characterized in that, The process of integrating the business logic with physical storage structure information to construct a knowledge graph includes: The extracted explicit metadata, non-explicit business logic relationships, data lineage information, and key database physical storage structure information are integrated to construct a knowledge graph; The nodes of the knowledge graph include tables, fields, records, devices, metrics, physical pages, and index nodes, and the edges of the knowledge graph include logical associations, physical associations, data lineages, and inheritance relationships.

4. The intelligent data quality repair method based on dynamic rule evolution as described in claim 3, characterized in that, Using graph neural networks to perform deep learning on the knowledge graph, and dynamically generating a global topological view based on the learning results of the graph neural network, includes: Graph neural networks learn node features through two layers of graph convolutional layers; node features include metadata, business logic features, data statistical features, and underlying physical storage attributes. Perform the first layer graph convolution operation to aggregate first-order neighbor information, apply the ReLU activation function to introduce non-linear characteristics, and apply the Dropout layer to randomly discard some node features to prevent overfitting. Perform a second-layer graph convolution operation to aggregate higher-order neighbor information and learn the logical-physical relationship patterns between nodes; Based on the global topology learned by the graph neural network, physical layer, logical layer, and application layer views are generated according to the target requirements.

5. The intelligent data quality repair method based on dynamic rule evolution as described in claim 4, characterized in that, This also includes: quantifying the importance of nodes in the constructed global topology graph to identify the core nodes with the greatest impact on the system; the importance quantification score is calculated as follows: ； in, For nodes Importance rating The damping coefficient is... Pointing to a node All nodes The set, For set Any node in, For nodes Importance rating For nodes The set of all outgoing chains, For nodes The degree of departure.

6. The intelligent data quality repair method based on dynamic rule evolution as described in claim 5, characterized in that, Constructing a multimodal association subgraph based on the aforementioned association anomaly includes: when a B-tree index root page is detected to be corrupted, constructing a multimodal association subgraph based on the B-tree index corruption; Based on the existing index structure meta-information in the knowledge graph, and combined with the traversal of the database page directory, all leaf pages belonging to the B-tree index are located and filtered out. Each leaf page contains a complete and ordered range of key values, the physical address of the actual data row, and bidirectional physical pointers between leaf pages. The leaf pages and the damaged index root pages are collectively abstracted into a multimodal association subgraph containing logical and physical relationships. The multimodal association subgraph is represented as follows: ； Among them, node set Including the root page node of the B-tree Intermediate page nodes Leaf page node and underlying data page nodes Each node Having feature vectors edge set This includes edges representing physical pointer links, doubly linked list edges between leaf pages, logical order relationships of records within a page, parent-child relationships between pages, and the binding relationship between checksums and page content.

7. The intelligent data quality repair method based on dynamic rule evolution as described in claim 6, characterized in that, Repairing structural defects in the multimodal association subgraph by inverting the correct physical structure through the graph neural network includes: the graph neural network aggregating information of its neighbors on the leaf page nodes; For each leaf page node Their neighbor information Calculated using the aggregation function AGG: ； node Its updated feature representation is as follows: ； in, For nodes Updated feature representation, and They are nodes and his neighbors The original characteristics, For nodes and his neighbors Features of the edges between them For nodes The set of neighboring nodes, For aggregate functions, For splicing operations, The weight matrix is ​​a learnable matrix; Using the aggregated leaf page feature set as input, a multi-layer graph neural network encoder is used to infer the correct structure of its parent node. The inference process is represented as follows: ； in, For the inferred root page or middle page High-level feature representation, For the graph neural network encoder model, It is the set of updated features of all leaf pages associated with the current parent node. It is the set of updated features of all intermediate pages associated with the current parent node; Based on the statistical distribution and sorting of the key value range of the leaf pages, and combined with the balance principle of the B-tree, the optimal key value split point is predicted; based on the physical address of the leaf page or intermediate page and their logical position after sorting, the correct physical subpage address pointed to by the root page is predicted. Based on the contextual information of surrounding healthy nodes and the learned patterns, the correct physical structure state of the damaged root page is corrected.

8. The intelligent data quality repair method based on dynamic rule evolution as described in claim 7, characterized in that, The reliability score of the repair results includes: A preset confidence threshold is used to score the reliability of the repair solution generated by the graph neural network in multiple dimensions. If the confidence score is not less than the confidence threshold, the result is deemed reliable, the repair is automatically executed, and high-quality data is output. If the confidence score is less than the confidence threshold, the repair is not executed, and a manual review process is triggered. Human experts combine their professional knowledge to make the final judgment and correction, and the results after human review and correction are stored and archived, forming a closed loop of structured data governance.

Images