An automated data verification and evaluation system based on multi-agent collaboration

Abstract

The application discloses an automatic data checking and evaluating system based on multi-agent cooperation, and relates to the technical field of data processing.The system comprises: a correlation anomaly judging unit, which is used for constructing a correlation anomaly detecting module, receiving multi-source heterogeneous data, performing attribute graph reconstruction, mapping to a unified embedding space, and judging and mining correlation checking rules; an optimal convergence threshold obtaining unit, which is used for determining an observation sequence and obtaining an optimal convergence threshold; a data checking unit, which is used for checking data to obtain multiple checking results; and a checking result judging unit, which is used for performing conflict judgment and resolution and visualizing.The application solves the technical problems of rule solidification, difficulty in dynamic adaptation, low multi-agent cooperation efficiency, high omission rate and inconsistent results in multi-source heterogeneous data checking in the prior art, and achieves the technical effects of improving the coverage, accuracy, automation level and multi-agent adaptive cooperation capability of data checking.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to an automated data verification and evaluation system based on multi-agent collaboration. Background Technology

[0002] In real-world business systems, heterogeneous data sources encompass databases, log files, API interfaces, IoT devices, and third-party platforms. These sources exhibit significant differences in format, semantics, granularity, and quality, posing a severe challenge to data verification and quality assessment. Traditional data verification and assessment methods, based on manually written fixed rules or automated scripts or rule engines with single verification logic, suffer from numerous shortcomings. Static rules struggle to dynamically adapt to business changes and data distribution shifts. When faced with new data anomalies or correlations, rules must be redefined, leading to delayed verification responses. Implicit relationships between multi-source data cannot be discovered through simple field-level rules, lacking the ability to mine and verify complex cross-source anomalies. In scenarios with large data volumes, multiple verification dimensions, and high real-time requirements, a single verification module or agent can easily become a performance bottleneck, struggling to cover multiple types of verification logic in parallel, resulting in low verification efficiency. Furthermore, systems employing multi-agent architectures lack adaptive collaboration mechanisms in task allocation, verification threshold setting, result fusion, and conflict resolution, and lack dynamic consensus negotiation, leading to inconsistent verification results, high false negative rates, and an inability to reach unified assessment conclusions.

[0003] Therefore, current technologies suffer from the technical problems of rigid rules and difficulty in dynamic adaptation during multi-source heterogeneous data verification, resulting in low efficiency of multi-agent collaboration, high false negative rate, and inconsistent results. Summary of the Invention

[0004] This application provides an automated data verification and evaluation system based on multi-agent collaboration, which solves the technical problems in the prior art of rigid rules and difficulty in dynamic adaptation in the verification of multi-source heterogeneous data, resulting in low efficiency, high false negative rate and inconsistent results of multi-agent collaboration. It achieves the technical effect of improving the coverage, accuracy, automation level and adaptive collaboration capability of multi-agent data verification.

[0005] This application provides an automated data verification and evaluation system based on multi-agent collaboration. The system includes: an association anomaly judgment unit, used to construct an association anomaly detection module based on a graph isomorphic network, which receives uploaded multi-source heterogeneous data, performs attribute graph reconstruction and maps it to a unified embedding space, and judges association anomalies based on the embedding vector distance of the same entity in the attribute graphs of different sources, and mines association verification rules; an optimal convergence threshold acquisition unit, used to determine the observation sequence, perform adaptive iterative convergence of multiple verification agents within the verification threshold interval, and obtain the optimal convergence threshold; a data verification unit, used to update agent parameters based on the optimal convergence threshold and add agents based on association verification rules to obtain a task agent cluster, which performs parallel data verification by performing task allocation and distribution on the multi-source heterogeneous data, and obtains multiple verification results; and a verification result determination unit, used to determine and resolve conflicts among the multiple verification results and visualize them on a terminal display interface.

[0006] In a possible implementation, receiving uploaded multi-source heterogeneous data and performing attribute graph reconstruction includes: the data access layer receiving multi-source heterogeneous data uploaded by the user terminal, preprocessing it to obtain multiple data sequences, wherein the preprocessing includes format standardization, missing value marking, and entity type inference; and modeling the multiple data sequences into an attribute graph after dividing them according to a single source, wherein the nodes in the attribute graph are fields, the edges are relationships between fields, the node attributes include field name, value, data type, and statistical characteristics, and the relationships between fields include primary and foreign key associations, temporal dependencies, and functional dependencies.

[0007] In a possible implementation, the association anomaly is determined by mapping the attribute graphs of the same entity to a unified embedding space and using the embedding vector distance between the attribute graphs of the same entity in different sources. This includes: using a graph isomorphic network to map the attribute graphs of different sources to a unified embedding space, performing graph isomorphic discrimination, and determining a first determination result, wherein whether the same entity is described is used as the discrimination target; in the unified embedding space, based on the first determination result, measuring the embedding vector distance between the attribute graphs of the same entity in different sources; if the embedding vector distance is greater than a dynamic threshold, it is determined to be an association anomaly, and an anomaly association score is marked in the unified embedding space.

[0008] In a possible implementation, mining association verification rules includes: identifying abnormal association markers and performing density clustering in the unified embedding space to mine association verification rules; wherein, the steps for mining association verification rules include: using abnormal association markers as a range, performing attribute graph clustering to mine common anomaly types; importing the common anomaly types into the rule generation module to generate association verification rules, and adding them to the verification library, wherein the rule generation module is an embedded module of the association anomaly detection module.

[0009] In a possible implementation, obtaining the optimal convergence threshold includes: constructing an observation sequence using the associated anomaly scores corresponding to each distance value in the embedding vector distance distribution; and performing adaptive convergence based on a verification threshold interval on multiple verification agents in parallel according to the observation sequence to determine the optimal convergence threshold, wherein each verification agent contains at least one optimal convergence threshold corresponding to its local verification logic.

[0010] In a possible implementation, multiple verification agents are subjected to adaptive convergence based on a verification threshold range in parallel, including: for the first verification agent, selecting a first verification threshold from a preset threshold range of the local verification logic; comparing the distance between the first verification threshold and the associated embedding vector to obtain a first associated anomaly determination result; comparing and statistically analyzing the first associated anomaly determination result with the corresponding associated anomaly score, and calculating the weighted sum of the false positive rate and the false negative rate as a first verification loss value; determining the gradient direction based on the first verification loss value, adjusting the first verification threshold to obtain a second verification threshold, and iterating through multiple rounds until the verification loss value is less than a preset loss value to determine the first optimal convergence threshold.

[0011] In a possible implementation, the task intelligence agent cluster includes: updating the parameters of multiple verification agents according to the optimal convergence threshold; deploying new verification agents as a task intelligence agent cluster by retrieving associated verification rules from the verification library; and decomposing the verification tasks of the multi-source heterogeneous data through a collaborative management module and distributing them to the task intelligence agent cluster in a distributed manner.

[0012] In a possible implementation, task allocation and distribution are performed, and data verification is conducted in parallel to obtain multiple verification results. This includes: performing distributed task allocation based on the data volume, verification complexity, and real-time load of the agent to obtain allocation results, wherein the allocation results contain multiple data sequence-task agent mappings; the task agent cluster receives the data sequence based on the allocation results, performs data verification, and obtains multiple verification results, wherein the multiple verification results correspond one-to-one with the task agent cluster.

[0013] In a possible implementation, it is determined whether there is a verification conflict among the multiple verification results. If so, a conflict resolution mechanism is triggered. The conflict resolution steps include: determining the verification weight based on the agent's historical verification accuracy; performing weighted voting on the conflict verification results according to the verification weight to obtain a valid verification result; and replacing the conflict verification results among the multiple verification results.

[0014] In possible implementations, visualization is performed on the terminal display interface, including: clustering and consolidating the multiple verification results by data primary key to generate a comprehensive verification report; displaying the anomaly details in a list view and the verification pass rate of each dimension in a radar chart in the comprehensive verification report, and displaying the results in a pop-up window on the terminal display interface.

[0015] This application proposes an automated data verification and evaluation system based on multi-agent collaboration. The system comprises: an anomaly detection unit for constructing an anomaly detection module, receiving multi-source heterogeneous data, performing attribute graph reconstruction and mapping to a unified embedding space, and mining association verification rules; an optimal convergence threshold acquisition unit for determining the observation sequence and obtaining the optimal convergence threshold; a data verification unit for performing data verification and obtaining multiple verification results; and a verification result determination unit for conflict determination, resolution, and visualization. This system solves the technical problems in existing multi-source heterogeneous data verification technologies, such as rigid rules that are difficult to dynamically adapt, leading to low efficiency, high false negative rates, and inconsistent results in multi-agent collaboration. It achieves the technical effect of improving the coverage, accuracy, automation level, and adaptive collaborative capabilities of multi-agent data verification. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a schematic diagram of an automated data verification and evaluation system based on multi-agent collaboration, provided as an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating the process of determining correlation anomalies in an automated data verification and evaluation system based on multi-agent collaboration, as provided in an embodiment of this application.

[0019] Figure labeling: 10 for correlation anomaly judgment unit, 20 for optimal convergence threshold acquisition unit, 30 for data verification unit, and 40 for verification result judgment unit. Detailed Implementation

[0020] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on the specific implementation methods, structures, features and effects of the present invention.

[0021] This application provides an automated data verification and evaluation system based on multi-agent collaboration, such as... Figure 1 As shown, the system includes: The association anomaly judgment unit 10 is used to construct an association anomaly detection module based on graph isomorphic network. By receiving uploaded multi-source heterogeneous data, it performs attribute graph reconstruction and maps it to a unified embedding space. It judges association anomalies based on the embedding vector distance of the same entity in attribute graphs of different sources and mines association verification rules.

[0022] Preferably, the association anomaly judgment unit constructs an association anomaly detection module by adopting a graph isomorphic network architecture, which unifies the structure of multi-source heterogeneous data. By utilizing the vector distance of the same entity in the multi-source graph, it automatically discovers inconsistencies in cross-source data and generates new association verification rules. Specifically, it receives multi-source heterogeneous data from different data sources such as databases, log files, and API interfaces. After preprocessing, it obtains multiple data sequences. The preprocessing includes format standardization, missing value marking, and entity type inference. Each data sequence is divided according to a single data source and modeled as an attribute graph. Its nodes are data fields, and its edges are relationships between fields. The node attributes include field name, field value, data type, and statistical characteristics.

[0023] Preferably, attribute graphs from different data sources are mapped to a unified embedding space to determine whether attribute graphs from different sources describe the same entity. In the unified embedding space, for attribute graphs from different sources that are determined to describe the same entity, the Euclidean distance between their embedding vectors is calculated. A smaller distance indicates that the two attribute graphs are closer in the embedding space, suggesting a more consistent representation of the same entity across different sources. A larger distance indicates that the two attribute graphs are more dispersed in the embedding space, suggesting significant differences in the representation of the same entity across different sources. This distance is then compared with a dynamic threshold. If the embedding vector distance is greater than the dynamic threshold, it is judged as an association anomaly, and an association score for this anomaly is marked in the unified embedding space. The dynamic threshold is determined by an adaptive iterative convergence mechanism. In the unified embedding space, density clustering is performed on the data points with marked anomaly association scores. Common anomaly types are identified through attribute graph clustering, and these common anomaly types are input into the rule generation module to generate association verification rules.

[0024] Furthermore, the association anomaly judgment unit 10 also includes a data access layer that receives multi-source heterogeneous data uploaded by the user terminal, preprocesses it to obtain multiple data sequences, wherein the preprocessing includes format standardization, missing value marking and entity type inference; and models the multiple data sequences into an attribute graph after dividing them according to a single source, wherein the nodes in the attribute graph are fields, the edges are relationships between fields, the node attributes include field name, value, data type and statistical characteristics, and the relationships between fields include primary and foreign key association, time-series dependency and functional dependency.

[0025] Preferably, the data access layer receives multi-source heterogeneous data uploaded by user terminals, which may come from different business systems, database tables, or log files. It preprocesses this multi-source heterogeneous data, including format standardization, missing value marking, and entity type inference. Format standardization unifies the representation of the same type of data from different data sources; missing value marking identifies missing or unknown parts of the data and marks them with specific symbols; entity type inference automatically identifies the data type stored in the fields and outputs multiple clearly structured data sequences. The multiple data sequences are then partitioned into single-source groups, grouping data sequences from the same data source together. Each group of data sequences is transformed into a graph structure representation, where nodes in the attribute graph represent fields, edges represent relationships between fields, and node attributes include field name, value, data type, and statistical characteristics. Relationships between fields include primary / foreign key associations, temporal dependencies, and functional dependencies.

[0026] Furthermore, such as Figure 2 As shown, the association anomaly judgment unit 10 further includes: using a graph isomorphic network to map the attribute graphs from different sources to a unified embedding space, performing graph isomorphic discrimination, and determining a first judgment result, wherein whether the same entity is described is used as the discrimination target; in the unified embedding space, based on the first judgment result, measuring the embedding vector distance of the same entity in the attribute graphs from different sources; if the embedding vector distance is greater than a dynamic threshold, it is judged as an association anomaly, and an abnormal association score is marked in the unified embedding space.

[0027] Preferably, a graph isomorphic network is a graph neural network architecture that can learn the topological structure and node attribute information of a graph, mapping the graph into fixed-length embedding vectors. The core characteristic of a graph isomorphic network is that for two isomorphic graphs, the output embedding vectors are the same or highly similar. Then, all attribute graphs are converted into embedding vectors through the same graph isomorphic network model and mapped to a unified embedding space. By comparing the embedding vectors of two attribute graphs, the discriminant is whether they describe the same entity, determining whether they belong to an isomorphic relationship in terms of structure and semantics, and determining the first judgment result. For example, determining whether the attribute graph of "Customer A" from the CRM system and the attribute graph of "Customer A" from the order system point to the same real customer. For attribute graph pairs determined to "describe the same entity", a distance metric is performed. Euclidean distance is used to calculate the distance between the embedding vectors of the same entity in attribute graphs from different sources. A small distance means that the graph representations of the same entity in different data sources are very close in the embedding space, indicating that the two data sources describe the entity consistently. A large distance means that the graph representations of the same entity in different data sources are far apart in the embedding space, indicating that the two data sources describe the entity significantly differently. If the embedding vector distance is greater than the dynamic threshold, it is judged as an association anomaly. An anomalous association score is marked in the unified embedding space to quantify the severity of the association anomaly.

[0028] Furthermore, the association anomaly judgment unit 10 also includes identifying anomaly association markers and performing density clustering processing in the unified embedding space to mine association verification rules; wherein, the steps for mining association verification rules include: using the anomaly association markers as a range, clustering through attribute graphs to mine common anomaly types; importing the common anomaly types into the rule generation module to generate association verification rules, and adding them to the verification library, wherein the rule generation module is an embedded module of the association anomaly detection module.

[0029] Preferably, within the unified embedding space, all data points marked as "abnormal associations" are selected and used as the clustering range. Within this range, density-based clustering is performed using the DBSCAN algorithm. Specifically, data points with similar distances and high density are grouped into the same cluster, while low-density, isolated data points are marked as noise. More specifically, using the abnormal association markings as the range, attribute graph clustering is used to analyze the common characteristics of the attribute graphs within each cluster. For example, all attribute graphs within a cluster may contain missing fields, have incorrect field value formats, or violate certain functional dependencies. The process involves identifying common anomaly types, which are the anomalous feature patterns shared by all attribute graphs within the same cluster. Each common anomaly type is then fed as raw input into the rule generation module for processing. This module transforms the pattern description of the "common anomaly type" into computer-executable structured validation rules. The rule generation module is an embedded module of the association anomaly detection module. For example, rules might state, "If the values ​​of field A and field B have a relationship of X, then mark it as an anomaly," or "If the difference in the representation of entity C in data source D and data source E exceeds a threshold T, then mark it as an association anomaly." Finally, the association validation rules are added to the validation library.

[0030] The optimal convergence threshold acquisition unit 20 is used to determine the observation sequence and perform adaptive iterative convergence of the verification threshold interval for multiple verification agents to obtain the optimal convergence threshold.

[0031] Furthermore, the optimal convergence threshold acquisition unit 20 also includes constructing an observation sequence based on the associated anomaly scores corresponding to each distance value in the embedded vector distance distribution; and performing adaptive convergence based on the verification threshold interval on multiple verification agents in parallel according to the observation sequence to determine the optimal convergence threshold, wherein each verification agent contains at least one optimal convergence threshold corresponding to the local verification logic.

[0032] Preferably, the observation sequence is constructed by matching the correlation anomaly scores corresponding to each distance value in the embedding vector distance distribution. That is, the embedding vector distance of each data point is paired with the correlation anomaly score to determine the observation sequence. Since the correlation of data in different batches is different, the standard for correlation verification needs to be adaptive. Based on the observation sequence, multiple verification agents are subjected to adaptive convergence based on the verification threshold range in parallel. In data verification, the higher the threshold is set, the easier it is to miss real anomalies, i.e., the false negative rate is high. The lower the threshold is set, the easier it is to misjudge normal data as anomalies, i.e., the false positive rate is high. Specifically, multiple verification agents execute their respective threshold convergence processes simultaneously and independently. Each agent adopts different local verification logic, such as format verification, field integrity verification, cross-source consistency verification, etc., to process the same observation sequence. Each verification agent presets a threshold value range, such as [0, 1], and iteratively adjusts the threshold to gradually approach the optimal value. After convergence, each verification agent obtains an optimal convergence threshold that minimizes the weighted sum of the false positive rate and the false negative rate of the verification agent. Each verification agent has at least one local verification logic and finally outputs the corresponding optimal convergence threshold.

[0033] Furthermore, the optimal convergence threshold acquisition unit 20 also includes: for the first verification agent, selecting a first verification threshold from a preset threshold range of the local verification logic; comparing the distance between the first verification threshold and the associated embedding vector to obtain a first association anomaly determination result; comparing and statistically analyzing the first association anomaly determination result with the corresponding association anomaly score to calculate the weighted sum of the false positive rate and the false negative rate as a first verification loss value; determining the gradient direction based on the first verification loss value, adjusting the first verification threshold to obtain a second verification threshold, and iterating through multiple rounds until the verification loss value is less than the preset loss value to determine the first optimal convergence threshold.

[0034] Preferably, the first verification agent is any one of multiple verification agents. The specific verification rule type that the verification agent is responsible for may be cross-source consistency verification, field integrity verification, format compliance verification, etc. A preset threshold range is configured for the verification logic, such as [0, 1]. An initial threshold is randomly selected from the preset threshold range as the first verification threshold. The distance of each embedding vector related to the verification agent is compared with the first verification threshold. If the embedding vector distance is greater than the first verification threshold, it is determined to be an association anomaly. If the embedding vector distance is less than or equal to the first verification threshold, it is determined to be a non-association anomaly. After comparing the distances of all embedding vectors, the first association anomaly determination result is obtained. The corresponding associated anomaly scores are binarized. For example, a score greater than 0.5 is considered a true anomaly, and a score less than or equal to 0.5 is considered a true normal score. The scores are then compared point by point with the first associated anomaly judgment result. The ratio of the number of data points judged as anomaly but actually normal to the total number of data points that are actually normal is calculated to determine the false positive rate. The ratio of the number of data points judged as normal but actually anomaly to the total number of data points that are actually anomaly is calculated to determine the false negative rate. The false positive rate and the false negative rate are then weighted and summed, with the weight coefficients adjusted according to business requirements, to obtain the first verification loss value.

[0035] Preferably, the current first verification loss value is compared with the previous first verification loss value, the trend of the loss value changing with the threshold is analyzed, and the gradient direction is determined. If the false negative rate dominates the loss value, the threshold is likely to be too high, and the gradient direction is negative. If the false positive rate dominates the loss value, the threshold is likely to be too low, and the gradient direction is positive. The first verification threshold is adjusted according to the gradient direction and the preset step size. The second verification threshold = first verification threshold + learning rate × gradient direction, and the second verification threshold is output. Through multiple rounds of iteration until the verification loss value is less than the preset loss value, where the preset loss value is the upper limit of tolerance set by the user or system, representing the maximum value of the weighted sum of the acceptable false positive rate and false negative rate, the first optimal convergence threshold is finally determined by convergence.

[0036] The data verification unit 30 is used to update the agent parameters based on the optimal convergence threshold and add agents based on the association verification rules to obtain a task agent cluster. By performing task allocation and distribution on the multi-source heterogeneous data, data verification is performed in parallel to obtain multiple verification results.

[0037] Furthermore, the data verification unit 30 also includes updating the parameters of multiple verification agents according to the optimal convergence threshold; deploying new verification agents as a task agent cluster by retrieving associated verification rules from the verification library; and decomposing the verification tasks of the multi-source heterogeneous data through the collaborative management module and distributing them to the task agent cluster in a distributed manner.

[0038] Preferably, the optimal convergence threshold of each verification agent is used as the update basis. The internal threshold parameters of each verification agent are updated to their corresponding optimal convergence thresholds, and multiple verification agents with updated parameters are output. At the same time, the associated verification rules of the verification library are read, and a new verification agent is created and deployed for each associated verification rule. Its local verification logic is the retrieved associated verification rule. The multiple verification agents with updated parameters and the newly deployed verification agents are merged to form a task agent cluster. Through the collaborative management module, the verification task of multi-source heterogeneous data is decomposed, including splitting the multi-source heterogeneous data to be verified into multiple sub-tasks. Each sub-task corresponds to a part of the data and is assigned to one or more verification agents. The decomposed sub-tasks are then sent to each agent in the task agent cluster in a distributed manner, and each agent performs data verification in parallel.

[0039] Furthermore, the data verification unit 30 also includes performing distributed task allocation based on the data volume, verification complexity, and real-time load of the intelligent agent to obtain allocation results, wherein the allocation results contain multiple data sequence-task intelligent agent mappings; the task intelligent agent cluster receives the data sequence based on the allocation results, performs data verification, and obtains multiple verification results, wherein the multiple verification results correspond one-to-one with the task intelligent agent cluster.

[0040] Preferably, the data volume refers to the number of data records contained in each data sequence. The larger the data volume, the longer it takes to process the sequence. Sequences with large data volumes should be assigned to agents with lower loads or higher processing capabilities. Verification complexity refers to the computational resource intensity required to perform verification on a data sequence. Different types of verification logic have different complexities. Verification with high complexity should be assigned to agents with sufficient computational resources. The real-time load of an agent is the workload currently undertaken by each verification agent, including the number of data sequences that have been assigned but not completed, the current CPU / memory utilization, network I / O, etc. Agents with high loads should be assigned fewer new tasks, while agents with low loads should be assigned more new tasks. The collaborative management module performs distributed task allocation based on the data volume, verification complexity, and real-time load of the agents. By prioritizing minimum load, weighted round-robin, or using the shortest expected processing time, it determines the verification agent that should be assigned to each data sequence, obtains the allocation results, and clarifies the correspondence between data sequences and verification agents, including multiple data sequence-task agent mappings. Then, each task agent obtains the original data sequence to be verified based on the data sequence identifier corresponding to that agent in the allocation result. Each task agent independently performs the verification operation on the data sequence it receives according to its local verification logic, and outputs multiple verification results, including verification pass / fail markers, exception details, exception scores, and other information. The multiple verification results correspond one-to-one with the task agent cluster.

[0041] The verification result determination unit 40 is used to determine and resolve conflicts among the multiple verification results and visualize them on the terminal display interface.

[0042] Preferably, the verification result determination unit is responsible for processing the verification results returned by multiple verification agents, including determining whether there are conflicts between the multiple verification results and resolving the conflicts, and finally visually displaying them on the terminal display interface. Here, verification conflict refers to different verification agents giving inconsistent judgment results for the same data sequence or the same data record. For example, agent M determines a certain data as "normal" while agent N determines the same data as "abnormal". All verification results are compared to identify whether there are different judgment results for the same data object.

[0043] Furthermore, the verification result determination unit 40 also includes determining whether there is a verification conflict among the multiple verification results. If so, a conflict resolution mechanism is triggered. The conflict resolution step includes: determining the verification weight based on the historical verification accuracy of the agent; performing weighted voting on the conflict verification results according to the verification weight to obtain a valid verification result; and replacing the conflict verification results among the multiple verification results.

[0044] Preferably, if multiple verification results are found to conflict, a conflict resolution mechanism is triggered. Specifically, each verification agent is assigned a weight value based on its historical performance in past verification tasks, such as accuracy. The higher the historical accuracy of the verification agent, the greater the weight. For the same data object that is in conflict, the judgment results of all agents on the object are collected, and a weighted vote is performed according to the weight of each agent. The judgment result with the highest number of votes is taken as the valid verification result. Then, the original conflict judgment result is replaced with the valid verification result, and the resolved verification result is output. Each data object without conflict corresponds to a unique verification judgment result.

[0045] Furthermore, the verification result determination unit 40 also includes: clustering and consolidating the multiple verification results according to the data primary key to generate a comprehensive verification report; displaying the anomaly details in a list view and the verification pass rate of each dimension in a radar chart in the comprehensive verification report, and displaying it in a pop-up window on the terminal display interface.

[0046] Preferably, multiple verification results are aggregated and organized according to the primary key of the data. This includes merging and organizing the scattered verification results according to the primary key of the data, summarizing all verification information belonging to the same data record, and obtaining a comprehensive verification report, including but not limited to an anomaly detail list, the pass rate of each dimension of verification, and the overall verification conclusion. Then, the report is visualized on the terminal display interface. This includes listing all data records judged as anomalies in a table format, including detailed information such as anomaly type, anomaly score, data source, and related fields; displaying the pass rates of different verification dimensions such as format verification, integrity verification, and cross-source consistency verification in a radar chart format on the same coordinate system, with each dimension corresponding to a coordinate axis, and the pass rate values ​​marked on the axes and connected to form a polygon; finally, the list view and radar chart are presented in a pop-up window on the terminal display interface to achieve visualization on the terminal display interface.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An automated data verification and evaluation system based on multi-agent collaboration, characterized in that, The system includes: The association anomaly judgment unit is used to construct an association anomaly detection module based on graph isomorphic networks. By receiving uploaded multi-source heterogeneous data, it performs attribute graph reconstruction and maps it to a unified embedding space. It judges association anomalies based on the embedding vector distance of the same entity in attribute graphs of different sources and mines association verification rules. The optimal convergence threshold acquisition unit is used to determine the observation sequence and perform adaptive iterative convergence of the verification threshold interval for multiple verification agents to obtain the optimal convergence threshold. The data verification unit is used to update the agent parameters based on the optimal convergence threshold and add agents based on the association verification rules to obtain a task agent cluster. By performing task allocation and distribution on the multi-source heterogeneous data, data verification is performed in parallel to obtain multiple verification results. The verification result determination unit is used to determine and resolve conflicts among the multiple verification results and visualize them on the terminal display interface. Receive uploaded multi-source heterogeneous data and perform attribute graph reconstruction, including: The data access layer receives multi-source heterogeneous data uploaded by the user terminal, and obtains multiple data sequences after preprocessing. The preprocessing includes format standardization, missing value marking, and entity type inference. Multiple data sequences are partitioned according to a single source and modeled as an attribute graph. In the attribute graph, nodes are fields, edges are relationships between fields, node attributes include field name, value, data type and statistical characteristics, and relationships between fields include primary and foreign key associations, time-series dependencies and functional dependencies. It is then mapped to a unified embedding space, and association anomaly detection is performed based on the embedding vector distance of the same entity in the attribute graphs of different sources, including: A graph isomorphic network is used to map attribute graphs from different sources to a unified embedding space, and graph isomorphic discrimination is performed to determine a first judgment result, wherein whether or not the same entity is described is used as the discrimination target; In the unified embedding space, based on the first determination result, the embedding vector distance of the same entity in the attribute graphs of different sources is measured; If the embedding vector distance is greater than the dynamic threshold, it is judged as an abnormal association, and an abnormal association score is marked in the unified embedding space. Discover association verification rules, including: In the unified embedding space, abnormal association markers are identified and density clustering is performed to mine association verification rules; The steps involved in mining association verification rules include: Using anomaly association markers as the scope, we can discover common anomaly types through attribute graph clustering. The common anomaly types are imported into the rule generation module to generate associated verification rules, which are then added to the verification library. The rule generation module is an embedded module of the associated anomaly detection module. To obtain the optimal convergence threshold, the following are included: The observation sequence is constructed using the associated anomaly scores corresponding to each distance value in the embedding vector distance distribution; Based on the observation sequence, multiple verification agents are subjected to adaptive convergence based on the verification threshold interval in parallel to determine the optimal convergence threshold. Each verification agent contains at least one optimal convergence threshold corresponding to the local verification logic. The task intelligence agent cluster includes: Based on the optimal convergence threshold, the parameters of multiple verification agents are updated. By retrieving associated verification rules from the verification library, new verification agents are deployed as a task agent cluster; The collaborative management module decomposes the verification tasks of the multi-source heterogeneous data and distributes them to the task intelligence agent cluster in a distributed manner.

2. The automated data verification and evaluation system based on multi-agent collaboration as described in claim 1, characterized in that, Parallel adaptive convergence based on a verification threshold interval is performed on multiple verification agents, including: For the first verification agent, a first verification threshold is selected from the preset threshold range of the local verification logic; The first verification threshold is compared with the distance of the associated embedding vector to obtain the first association anomaly determination result; The first associated anomaly determination result is compared and statistically analyzed with the corresponding associated anomaly score, and the weighted sum of the false alarm rate and the false negative rate is calculated as the first verification loss value. Based on the first verification loss value, the gradient direction is determined, the first verification threshold is adjusted to obtain the second verification threshold, and through multiple iterations until the verification loss value is less than the preset loss value, the first optimal convergence threshold is determined.

3. The automated data verification and evaluation system based on multi-agent collaboration as described in claim 1, characterized in that, The task allocation and distribution are executed, and data verification is performed in parallel to obtain multiple verification results, including: Based on the data volume, verification complexity, and real-time load of the agent, a distributed task allocation is performed to obtain the allocation result, which includes multiple data sequence-task agent mappings. The task intelligence agent cluster receives a data sequence based on the allocation result, performs data verification, and obtains multiple verification results, wherein each of the multiple verification results corresponds one-to-one with the task intelligence agent cluster.

4. The automated data verification and evaluation system based on multi-agent collaboration as described in claim 1, characterized in that, Determine whether there is a verification conflict among the multiple verification results; if so, trigger the conflict resolution mechanism. The conflict resolution steps include: The verification weight is determined by the historical verification accuracy of the agent; Based on the verification weights, the conflict verification results are weighted and voted on to obtain valid verification results, and the conflict verification results among the multiple verification results are replaced.

5. The automated data verification and evaluation system based on multi-agent collaboration as described in claim 1, characterized in that, Visualization is performed on the terminal display interface, including: The multiple verification results are aggregated and consolidated according to the data primary key to generate a comprehensive verification report; The comprehensive verification report displays anomaly details in a list view, shows the pass rate of each dimension in a radar chart, and displays a pop-up window on the terminal display interface.

Images