Method and apparatus for optimized control of single-machine graph data cleaning, and device and medium

By decomposing the binary star pattern rules into single-star components and combining them with a pipelined parallel model of CPU/GPU heterogeneous computing resources, the single-machine graph data cleaning process is optimized, solving the problem of insufficient resource utilization and improving computing efficiency.

WO2026232626A1PCT designated stage Publication Date: 2026-11-12SHENZHEN INST OF COMPUTING SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/094525
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2025-05-13
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing single-machine graph data cleaning systems suffer from resource idleness and unsaturated parallelism when handling complex graph cleaning tasks, failing to fully utilize heterogeneous computing resources and resulting in low computational efficiency.

Method used

The binary star pattern rules are decomposed into single-star components. Single-star beams are constructed by intersection and union operations of single-star components. The process of graph data cleaning is optimized by using heterogeneous computing resources of CPU/GPU for pipelined parallelism and single instruction multiple data parallelism.

Benefits of technology

It effectively avoids redundant calculations in the pattern matching stage, improves the system's execution efficiency, and significantly enhances performance, especially in large-scale graph data cleaning tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025094525_12112026_PF_FP_ABST
    Figure CN2025094525_12112026_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to a method and apparatus for optimized control of single-machine graph data cleaning, and a device and a medium. The method comprises: acquiring target graph data and a rule set; decomposing all dual-star pattern rules in the rule set into single-star components, and determining a component structure of each single-star component; performing intersection and union operations on single-star components, component structures of which have nodes and edges in common, in order to obtain single-star bundles each comprising a common sub-structure and a complete structure; and on the basis of each single-star bundle, performing matching on the target graph data to obtain corresponding matching entities, using the rule set to perform enumeration-based verification on all the matching entities, in order to obtain conflicting matching-entity pairs, and performing conflict repair on the conflicting matching-entity pairs, in order to obtain a graph data cleaning result. Dual-star pattern rules are decomposed, single-star bundles are constructed from single-star components obtained from the decomposition, and entity matching is executed on the basis of each single-star bundle, thereby eliminating the need for repeated computations on shared structures, effectively avoiding redundant computations in a pattern matching phase, and thus improving the execution efficiency of the system.
Need to check novelty before this filing date? Find Prior Art

Description

Optimized control methods, devices, equipment and media for single-machine image data cleaning

[0001] This application is based on and claims priority to Chinese Invention Application No. 202510596651.X, filed on May 9, 2025, entitled "Optimization Control Method, Apparatus, Equipment and Medium for Single-Machine Graphic Data Cleaning". Technical Field

[0002] This application relates to the field of big data technology, and in particular to an optimized control method, apparatus, equipment and medium for single-machine graph data cleaning. Background Technology

[0003] In recent years, with the advancement of social informatization, the scale of graph data generated or collected by computer systems is rapidly expanding. For example, Facebook's social network has nearly 2 billion users, and modeling it as a graph would contain nearly 2 billion nodes and an order of magnitude higher number of edges, posing a severe challenge to the performance of graph data cleaning systems. However, in certain scenarios, such as the Industrial Internet of Things (IIoT), computing devices are typically deployed at the edge, making distributed cluster deployment difficult to achieve, while cloud-based computing models face issues such as cost and privacy, leading to an increasing demand for stand-alone systems.

[0004] Considering the practical needs for large-scale graph data cleaning and resource-constrained use cases, various related solutions have been proposed in recent years, which can be broadly categorized into single-machine general-purpose graph computing systems, machine learning-based graph cleaning systems, and graph data cleaning systems based on graph cleaning rules. Single-machine general-purpose graph computing systems typically rely on a single parallel model to optimize the efficiency of specific types of graph computing tasks (e.g., connected component computation, shortest path query). However, for more complex graph cleaning tasks, these systems cannot fully utilize heterogeneous computing resources, often resulting in resource idleness and GPU thread bundle divergence, leading to overall performance degradation. Machine learning-based graph data cleaning systems rely on large amounts of labeled data and model hyperparameter tuning, but their prediction results lack guarantees of accuracy and interpretability, making them unsuitable for scenarios with insufficient data labeling or high requirements for result reliability. Graph cleaning systems based on graph cleaning rules require applying all rules within the graph cleaning rule set to the data graph. This process involves frequent graph pattern matching and pairwise verification of graph pattern entities, resulting in significant computational overhead. Furthermore, due to the overlap of patterns and attributes among the input rules, sequential application of rules leads to a large amount of redundant computation.

[0005] Single-machine graph data cleaning systems, particularly rule-based graph data cleaning, involve numerous computationally intensive tasks such as graph pattern matching and entity pair enumeration. Existing single-machine systems, with their limited computing resources, struggle to support large-scale graph data cleaning tasks. Graph data cleaning requires deriving all rules within the input set of graph cleaning rules. This process involves a significant amount of repetitive computation, further exacerbating performance bottlenecks. Furthermore, the use of single parallel models, such as those centered on nodes / edges / subgraphs, suffers from resource idleness and unsaturated parallelism, failing to fully utilize computing resources.

[0006] Therefore, optimizing the data cleaning process for single-machine graphs to improve computational efficiency has become an urgent problem to be solved. Summary of the Invention

[0007] In view of this, embodiments of this application provide an optimized control method, apparatus, device, and medium for single-machine graph data cleaning, in order to solve the problem of how to optimize the processing of single-machine graph data cleaning to improve computational efficiency.

[0008] In a first aspect, embodiments of this application provide an optimized control method for single-machine graph data cleaning, including:

[0009] A set of rules is formed by acquiring target graph data and binary star pattern rules obtained by rule discovery based on the target graph data. The binary star pattern rules represent the association and dependency relationship between any two different entities.

[0010] All binary star pattern rules in the rule set are decomposed into single-star components. The component structure of each single-star component is determined. Each single-star component includes a predicate condition that an entity must satisfy and an attribute to be materialized. The component structure includes at least one node and an edge connecting two nodes.

[0011] The single-star components with the same nodes and edges are subjected to intersection and union operations to obtain a single-star bundle including a common substructure and a complete structure. The common substructure includes shared node connections and shared predicate conditions, and the complete structure includes all node connections and all materialized attributes of all merged single-star components.

[0012] Based on each single star beam, the corresponding matching entity is matched from the target graph data. The rule set is used to enumerate and verify all matching entities to obtain conflicting matching entity pairs. The conflicting matching entity pairs are then repaired to obtain the graph data cleaning result.

[0013] Secondly, according to an embodiment of this application, an optimized control device for single-machine graph data cleaning includes:

[0014] The data acquisition module is used to acquire target graph data and a set of rules consisting of binary star pattern rules obtained by rule discovery based on the target graph data. The binary star pattern rules represent the association and dependency relationship between any two different entities.

[0015] The pattern decomposition module is used to decompose all binary star pattern rules in the rule set into single-star components, determine the component structure of each single-star component, and the single-star component includes a predicate condition that an entity must satisfy and an attribute to be materialized. The component structure includes at least one node and an edge connecting two nodes.

[0016] The pattern combination module is used to perform intersection and union operations on single-star components with the same nodes and edges to obtain single-star bundles including common substructures and complete structures. The common substructures include shared node connection relationships and shared predicate conditions, and the complete structure includes all node connection relationships and all materialized attributes of all merged single-star components.

[0017] The graph data cleaning module is used to match corresponding entities from the target graph data based on each single star beam, enumerate and verify all matching entities using the rule set, obtain conflicting matching entity pairs, perform conflict repair on the conflicting matching entity pairs, and obtain the graph data cleaning result.

[0018] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the optimized control method for single-machine graph data cleaning as described in the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the optimized control method for single-machine graph data cleaning as described in the first aspect.

[0020] The beneficial effects of the embodiments of this application compared with the prior art are:

[0021] This application acquires target graph data and a rule set consisting of binary star pattern rules obtained through rule discovery based on the target graph data. All binary star pattern rules in the rule set are decomposed into single-star components. The component structure of each single-star component is determined. Single-star components with the same nodes and edges are subjected to intersection and union operations to obtain single-star bundles including common substructures and complete structures. Based on each single-star bundle, corresponding matching entities are matched from the target graph data. All matching entities are enumerated and verified using the rule set to obtain conflicting matching entity pairs. Conflict repair is performed on these conflicting matching entity pairs to obtain the graph data cleaning result. By decomposing the binary star pattern rules and constructing single-star bundles from the decomposed single-star components, and executing matching entities with single-star bundles, redundant calculations on shared structures are avoided, effectively preventing redundant calculations in the pattern matching stage and improving the system's execution efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 is a schematic diagram of an application environment for an optimized control method for single-machine graph data cleaning provided in Embodiment 1 of this application;

[0024] Figure 2 is a flowchart illustrating an optimized control method for single-machine graph data cleaning provided in Embodiment 2 of this application;

[0025] Figure 3 is an example diagram of graph data and rules in graph data cleaning according to Embodiment 2 of this application;

[0026] Figure 4 is a schematic diagram of a graph data cleaning workflow provided in Embodiment 2 of this application;

[0027] Figure 5 is a data example diagram of a single-star component combination provided in Embodiment 2 of this application;

[0028] Figure 6 is a schematic diagram of the architecture of a single-machine graph data cleaning system provided in Embodiment 2 of this application;

[0029] Figure 7 is a flowchart illustrating an optimized control method for single-machine graph data cleaning provided in Embodiment 3 of this application;

[0030] Figure 8 is a schematic diagram of the structure of an optimized control device for single-machine image data cleaning provided in Embodiment 4 of this application;

[0031] Figure 9 is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] To illustrate the technical solution of this application, specific embodiments are described below.

[0034] This application provides an optimized control method for single-machine graph data cleaning, applicable to the environment shown in Figure 1. The client communicates with the server, the optimized control method is executed on the server, and the client initiates graph cleaning rule matching by sending a corresponding start command to the server. The server can store graph data, which can be directly accessed during graph cleaning rule matching. The server is equipped with a storage disk, which provides memory support for graph cleaning rule matching.

[0035] The client side includes, but is not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server side can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0036] Referring to Figure 2, which is a flowchart of an optimized control method for single-machine graph data cleaning provided in Embodiment 2 of this application, the optimized control method for single-machine graph data cleaning can be applied to the server in Figure 1. The server performs optimized control when matching graph cleaning rules, specifically for the optimized control of data processing during the matching process of graph cleaning rules.

[0037] As shown in Figure 2, the optimized control method for single-machine graph data cleaning may include the following steps:

[0038] Step S201: Obtain the target graph data and the rule set consisting of the binary star pattern rules obtained by rule discovery based on the target graph data.

[0039] In this embodiment, the binary star pattern rule represents the association and dependency relationship between any two different entities. The target graph data can be stored on a disk for later retrieval, and correspondingly, the rule set can also be stored on the disk. Of course, the rules in the rule set can be discovered from the target graph data.

[0040] The process of matching the image cleaning rules in this application is as follows:

[0041] Figure 3 shows an example of graph data and rules in graph data cleaning according to Embodiment 2 of this application. The mathematical form of the graph cleaning rules can be expressed as: Here, Q is a binary star graph pattern used to identify related entities in the graph, X is a conjunction of a set of predicates, and p0 is a predicate; X→p0 can represent the association and dependency relationships between entities in the graph. The rules... First, a binary star pattern Q is matched in graph G. Then, it is asserted that for each entity in the binary star pattern, if the machine learning model Ms considers the conference and author names of the two paper nodes to be consistent, and one paper belongs to the CS category, then the other paper should also belong to the CS category. This mathematical form based on logical reasoning guarantees the correctness and interpretability of the graph data cleaning results. In addition, the graph cleaning rules are compatible with machine learning models in the form of predicates, thus the data cleaning results also have strong generalization ability. Due to its strong theoretical guarantee, more and more graph data cleaning systems are choosing graph cleaning rules as their theoretical foundation.

[0042] As shown in Figure 4, the graph data cleaning based on graph cleaning rules in this embodiment includes three stages: (1) The rule discovery stage enumerates graph cleaning rules according to the user configuration and applies the enumerated rules to the data graph sample input by the user. If the support and confidence of the enumerated rules are both higher than the threshold set by the user, they will be placed into the output set Σ. After all the enumerated rules have been applied, the rule set Σ will be output as the result. (2) The error detection stage accepts graph G and the output Σ of the previous stage as input and applies these rules to the data graph: first, it matches the binary star pattern defined by the graph cleaning rules in the graph, and then verifies the dependency conditions defined by the rules on the matching entities. Entities that do not meet the dependency will be output as conflicts. (3) The error repair stage accepts graph G, rule set Σ, and conflict data from the previous stage as input, repairs the errors in the graph data, and returns to the second stage to perform error detection again. When there is no conflict data output in the second stage, the process terminates.

[0043] In the workflow described above, matching graph cleaning rules is crucial throughout the graph data cleaning process. Therefore, optimizing the matching of graph cleaning rules is essential for improving the overall performance of the graph data cleaning system.

[0044] Step S202: Decompose all binary star pattern rules in the rule set into single-star components and determine the component structure of each single-star component.

[0045] Among them, the dual-star pattern rule is a rule for two entities, while the single-star component is a rule for one entity. The dual-star pattern rule can be decomposed into two single-star components. There are shared substructures in the single-star components. Therefore, the shared substructure can be processed once without multiple processing.

[0046] The binary star pattern in the rules is broken down into several independent single-star components. in This is a single-star graph pattern. X defines the predicate conditions that the single-star component must satisfy, and FQ defines the property that the single-star component needs to be materialized. Single-star components with shared substructures can be combined into single-star bundles and matched as a whole, thus avoiding repeated matching of shared substructures. Therefore, the single-star component includes the predicate conditions that an entity must satisfy and the property to be materialized. The component structure includes at least one node and an edge connecting two nodes.

[0047] In this embodiment, the matching of the graph cleaning rule includes two stages: (1) Single-star graph pattern matching. In the graph cleaning task, there are shared substructures among the rules in the rule set (e.g., shared star sub-patterns, shared predicates). In this stage, the binary star pattern of the graph cleaning rule is decomposed into two single-star components for matching, which can avoid the excessive intermediate results caused by directly materializing the binary star pattern. (2) Binary star matching enumeration. In this stage, each pair is enumerated and checked to see if the matching of the two single-star patterns corresponding to the graph cleaning rule satisfies the predicate dependency defined by the rule. Entity pairs that do not satisfy the dependency will be regarded as data conflicts.

[0048] Each graph cleaning rule is broken down into two single-star components. Then, single-star components sharing a substructure are combined into a single-star bundle. Finally, the single-star bundles are matched as a whole. Figure 5 shows a data example diagram of single-star component combination provided in Embodiment 2 of this application. In this example, the binary pattern of the rule can be broken down into... as well as There are two single-star components that share three nodes: x0, x1, and x2, as well as two edges: x0→x1 and x0→x2.

[0049] Step S203: Perform intersection and union operations on single-star components with the same nodes and edges to obtain a single-star bundle including common substructures and a complete structure.

[0050] The common substructure includes shared node connection relationships and shared predicate conditions, and the complete structure includes all node connection relationships and all materialized attributes of all merged single-star components.

[0051] As shown in Figure 5, through the intersection operator Extract the common substructure, including the shared subpattern Q(l*r) and the shared predicate Xl∩Xr. This common substructure can be regarded as a filter; any single-star match that does not satisfy the shared subpattern or the shared predicate cannot be a match of any single-star component that makes up the single-star bundle, and therefore can be safely filtered.

[0052] For entities that satisfy the filtering conditions defined by the common substructure, the complete structure is extracted using the union operator ⊕, including the complete graph pattern Q(l+r) and the complete predicate set defined by FQ(l+r), and the entity is materialized accordingly. This materialization result can be regarded as a superset of all single-star pattern entities that make up the single-star bundle, and therefore can be equivalent to the matching of any single-star component that makes up the single-star bundle.

[0053] Step S204: Based on each single star beam, match the corresponding matching entity from the target graph data, use the rule set to enumerate and verify all matching entities, obtain conflicting matching entity pairs, perform conflict repair on the conflicting matching entity pairs, and obtain the graph data cleaning result.

[0054] Specifically, for any single star beam, the corresponding matching entity is matched from the target graph data, which is the target data in the target graph data. For each matching entity, the binary star pattern rule in the rule set mentioned above is used to enumerate and pair any two matching entities. If there is a conflict, it indicates that the matching entity pair is a conflicting matching entity pair. The conflicting matching entity pair is then repaired to obtain the graph data cleaning result.

[0055] This application's embodiments utilize CPU / GPU heterogeneous computing resources. To fully utilize these resources, this application proposes a hybrid parallel model for data cleaning tasks, including: pipelined parallelism, single instruction multiple data (SIMD) parallelism, and dependency-free task parallelism.

[0056] The pipeline system architecture is as follows:

[0057] Figure 6 shows a schematic diagram of the architecture of a single-machine graph data cleaning system provided in Embodiment 2 of this application. The present invention divides the graph data cleaning task into two stages, as follows:

[0058] 1. Single-beam matching stage: (a) The data loader loads graph data from the SSD into memory; (b) The query parser parses the query parameters input by the user. If the task type is rule discovery, the rule generator enumerates and generates a rule set; otherwise, the rule parser loads and parses the rule set; (c) Subsequently, the pattern combiner reorganizes the rules in the rule set into single beams; (d) The pattern matcher matches single-beam entities in the data graph; (e) To reduce the computational load of subsequent matching enumeration, the matched entities are placed into buckets by the entity bucketer according to their materialized properties. Since cross-bucket entities have attribute differences, they cannot form a pair of binary patterns. Therefore, the subsequent binary matching stage can only enumerate entities located in the same bucket; (f) For large-scale data graphs, the matched entities may not be able to be completely placed in memory. Therefore, this embodiment introduces a buffer manager to manage the matched single-beam entities. The part exceeding memory will be swapped into the swap space in the SSD; (g) Finally, the matched entities in each bucket are submitted to the GPU via the GPU agent as input for the next stage task.

[0059] 2. Dual-Star Matching Enumeration Phase: (a) The GPU agent first loads the single-star beam entities matched in the previous phase into GPU memory; (b) Each graph cleaning rule in the rule set corresponds to an enumeration task. The dual-star matching enumerator allocates a thread block for each enumeration task and then verifies whether each pair of single-star beam entities satisfies the dependencies specified by the graph cleaning rules in a single instruction multiple data (SID) manner. All entity pairs that violate the rule dependencies are marked as conflicting data; (c) After all single-star beam entity pairs have been enumerated, the conflicting data is copied back to memory via the GPU agent; (d) Finally, the result aggregator repairs data conflicts and saves the results to the SSD.

[0060] Pipeline Parallelism: Considering the numerous branch judgments involved in the pattern matching stage in the first stage, which could easily lead to thread bundle divergence in single-instruction multiple-data parallel mode, this stage utilizes multi-core CPU parallelism for acceleration. The second stage has a more regular computational logic but involves O(N^2) complexity enumeration, resulting in high computational overhead; therefore, GPU acceleration is used. This invention, through a pipelined approach, overlaps the CPU computation, GPU computation, and data transfer overhead between the two stages, effectively improving system performance.

[0061] A hybrid parallel model was designed for graph data cleaning tasks. By using pipelined parallelism between CPU and GPU, the computation of each stage of the algorithm and the data transfer overhead between CPU and GPU are effectively overlapped. Single instruction multiple data parallelism effectively utilizes the powerful parallel capabilities of GPU. Dependency-free parallelism further improves the resource utilization of GPU.

[0062] Optionally, after matching the corresponding entity from the target map data based on each individual star beam, the method further includes:

[0063] Based on the attribute to be materialized, all matching entities are bucketed to obtain N entity buckets. Matching entities with the same attribute to be materialized are in the same entity bucket, where N is a positive integer.

[0064] The step of enumerating and verifying all matching entities using the rule set to obtain conflicting matching entity pairs includes:

[0065] For any entity bucket, the rule set is used to enumerate and verify all matching entities within the entity bucket to obtain the conflicting matching entity pairs corresponding to the entity bucket.

[0066] Iterate through all entity buckets to obtain all conflicting matching entity pairs.

[0067] Among them, setting up entity buckets can group matching entities with the same objectification attributes into one bucket. For objectified entities in an entity bucket, they can be paired by the binary star pattern rule. However, entities across buckets have attribute differences and cannot form a binary star pattern. Therefore, the subsequent binary star matching stage can only enumerate entities located in the same bucket.

[0068] Optionally, an enumeration task is performed using a GPU, wherein the rule set is used to enumerate and verify all matching entities within the entity bucket to obtain conflicting matching entity pairs corresponding to the entity bucket, including:

[0069] In the GPU, a thread block is allocated for each binary star pattern rule in the rule set. The thread block is used to execute the enumeration task of the corresponding binary star pattern rule.

[0070] For any thread block, in the form of single instruction multiple data, all matching entities in the entity bucket are checked pairwise to detect whether the dependencies specified by the corresponding binary star pattern rule are met.

[0071] If the dependencies specified by the corresponding binary star pattern rule are not met, then the corresponding entity pair is determined to be a conflicting matching entity pair.

[0072] In this embodiment, the single-instruction multiple-data (SIMDG) capability of the GPU is fully utilized. During the dual-star matching enumeration stage, the embodiment evenly binds single-star beam entities to different GPU threads based on the task ID and single-star beam entity ID. In the parallel mode of SIMDG, this system can enumerate thousands of single-star beam entity pairs simultaneously. Since this process rarely involves branch judgments, the powerful parallel capabilities of the GPU can be fully utilized.

[0073] Furthermore, in this embodiment, different binary star matching enumeration tasks are assigned to different thread blocks of the GPU, forming dependency-free task parallelism. Since the binary star matching enumeration tasks are independent of each other, this application can adaptively submit enumeration tasks to the GPU. For example, if the current enumeration task volume is small and the GPU load is not saturated, this invention will submit other enumeration tasks to idle thread blocks to fully utilize GPU resources.

[0074] Optionally, the CPU is used to perform the single-satellite beam matching task. After binning all matching entities according to the attributes to be materialized to obtain N entity bins, the following steps are also included:

[0075] The matching entities in the N entity buckets are placed in the buffer of the CPU. If the buffer manager detects that the matching entities exceed the memory capacity of the buffer, the excess part is swapped into the swap area of ​​the storage area storing the target graph data.

[0076] The matching entities for each entity bucket in the buffer are submitted to the GPU via a GPU agent.

[0077] Among them, the single-machine graph data cleaning system shown in Figure 6 achieves the exchange of entities and results through the cooperation of buffer management and GPU agent, thereby improving the efficient interaction between CPU and GPU.

[0078] Optionally, after traversing all entity buckets and obtaining all conflicting matching entity pairs, the process also includes:

[0079] The GPU agent stores all conflicting matching entities into the CPU.

[0080] The process of conflict repair on the conflicting matched entity pairs to obtain graph data cleaning results includes:

[0081] The result aggregator is used to perform conflict repair on the conflicting matching entities in the CPU to obtain the graph data cleaning result.

[0082] The results of the graph data cleaning are saved to the storage area where the target graph data is stored.

[0083] As shown in Figure 6, the single-machine graph data cleaning system uses a GPU agent to store conflicting matching entities in the CPU. The CPU then uses a result aggregator to repair the conflicting matching entities, thereby obtaining the data cleaning result.

[0084] Optionally, the rule set consisting of the target map data and the binary star pattern rules obtained by rule discovery based on the target map data includes:

[0085] The CPU retrieves the target map data from the storage area where the target map data is stored.

[0086] Rule discovery is performed on the target graph data to obtain at least one binary star pattern rule, and all binary star pattern rules are combined into a rule set.

[0087] In the single-machine graph data cleaning system architecture shown in Figure 6, the target graph data is stored on an SSD disk, and the corresponding rule set is also stored on an SSD disk.

[0088] This application embodiment acquires target graph data and a rule set consisting of binary star pattern rules obtained through rule discovery based on the target graph data. All binary star pattern rules in the rule set are decomposed into single-star components. The component structure of each single-star component is determined. Single-star components with the same nodes and edges are subjected to intersection and union operations to obtain single-star bundles including common substructures and complete structures. Based on each single-star bundle, corresponding matching entities are matched from the target graph data. All matching entities are enumerated and verified using the rule set to obtain conflicting matching entity pairs. Conflict repair is performed on these conflicting matching entity pairs to obtain the graph data cleaning result. By decomposing the binary star pattern rules and constructing single-star bundles from the decomposed single-star components, and using single-star bundles to execute matching entities, redundant calculations on shared structures are avoided, effectively preventing redundant calculations in the pattern matching stage and improving the system's execution efficiency.

[0089] Referring to Figure 7, which is a flowchart illustrating an optimized control method for single-machine graph data cleaning according to Embodiment 3 of this application, as shown in Figure 7, step S202 above, which involves uniformly dividing the memory segment based on the number of buckets for matching entities obtained from the preset graph data using the entity bucketing device, to obtain memory slots corresponding to the number of buckets, may include the following steps:

[0090] Step S701: Monitor the GPU task queue to obtain the real-time GPU task queue length.

[0091] Step S702: When the length of the real-time GPU task queue is less than or equal to the first threshold, enable the CPU to combine all single-star components that meet the combination requirements when performing the intersection and union operations of single-star components that include the same nodes and edges in the component structure to obtain a single-star bundle including a common substructure and a complete structure.

[0092] Step S703: If the length of the real-time GPU task queue is less than or equal to the second threshold and greater than the first threshold, enable the CPU to combine all single-star components that meet the combination requirements and whose common substructure is greater than the preset structure size when performing the intersection and union operations of single-star components that include the same nodes and edges in the component structure to obtain a single-star bundle including a common substructure and a complete structure.

[0093] Step S704: When the length of the real-time GPU task queue is greater than the second threshold, the CPU is prohibited from executing the intersection and union operations of single-star components that include the same nodes and edges in the component structure to obtain a single-star bundle including the common substructure and the complete structure, and the single-star component to be processed is directly output as a single-star bundle.

[0094] In this embodiment, different graph data cleaning workloads exist in real-world scenarios, such as CPU-intensive and GPU-intensive workloads. If the computational load on the CPU and GPU cannot be effectively balanced, the pipelined parallel model will degenerate into a sequential model. Specifically, when more rules are combined to form a single beam, the repetitive matching work required on the CPU side is reduced; however, since the set of single beam instances is a superset of the set of all single beam component instances, enumerating single beam instances introduces greater computational overhead on the GPU side. Therefore, rule bundling can be seen as a means of shifting CPU workload to the GPU.

[0095] Among them, a GPU task queue monitor was set up, and three rule combination modes were set according to different task loads.

[0096] 1. Greedy Mode: When the GPU task queue length is less than or equal to the first threshold π1, the monitor considers the current workload to be CPU-intensive, and the system enters greedy mode. In this mode, the system combines all possible rules to push the workload from the CPU side to the GPU.

[0097] 2. Balanced Mode: When the GPU task queue length is less than or equal to the second threshold π2 but greater than the first threshold π1, the monitor considers the current CPU and GPU workloads to be relatively balanced, and therefore will prioritize combining rules with larger shared substructures.

[0098] 3. Disabled mode: When the GPU task queue length is greater than the second threshold π2, the monitor considers the current workload to be GPU intensive, and therefore disables rule combination optimization to prevent CPU load from being pushed to the GPU.

[0099] For this application, on the Semantic Scholar dataset, a large graph with 160 million nodes and 750 million edges, it outperforms state-of-the-art systems in both error detection and error repair tasks. For a state-of-the-art baseline distributed system with a 32-node configuration, this application achieves at least an 8.09x performance improvement.

[0100] This application's embodiments address different graph data cleaning workloads. By monitoring the GPU task queue, the present invention adaptively employs multiple rule combination modes to balance the computational load of the CPU and GPU, thereby improving system performance.

[0101] Corresponding to the single-machine graph data cleaning optimization control method in the above embodiments, Figure 8 shows a structural block diagram of the single-machine graph data cleaning optimization control device provided in Embodiment 4 of this application. The above-mentioned single-machine graph data cleaning optimization control device can be applied to the server in Figure 1. The server performs optimization control when performing graph cleaning rule matching, specifically targeting the optimization control of data processing during the graph cleaning rule matching process. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0102] Referring to Figure 8, the optimized control device for single-machine image data cleaning includes:

[0103] The data acquisition module 81 is used to acquire target graph data and a set of rules consisting of binary star pattern rules obtained by rule discovery based on the target graph data. The binary star pattern rules represent the association and dependency relationship between any two different entities.

[0104] The pattern decomposition module 82 is used to decompose all binary star pattern rules in the rule set into single-star components, determine the component structure of each single-star component, and the single-star component includes a predicate condition that an entity must satisfy and an attribute to be materialized. The component structure includes at least one node and an edge connecting two nodes.

[0105] The pattern combination module 83 is used to perform intersection and union operations on single-star components with the same nodes and edges in the component structure to obtain a single-star bundle including a common substructure and a complete structure. The common substructure includes shared node connection relationships and shared predicate conditions, and the complete structure includes all node connection relationships and all materialized attributes of all merged single-star components.

[0106] The graph data cleaning module 84 is used to match the corresponding matching entity from the target graph data based on each single star beam, enumerate and verify all matching entities using the rule set, obtain conflicting matching entity pairs, perform conflict repair on the conflicting matching entity pairs, and obtain the graph data cleaning result.

[0107] Optionally, the optimized control device for single-machine map data cleaning also includes:

[0108] After matching the corresponding entity from the target map data based on each single star beam, all matching entities are bucketed according to the property to be materialized, resulting in N entity buckets. Matching entities with the same property to be materialized are in the same entity bucket, and N is an integer greater than zero.

[0109] The graph data cleaning module 84 includes:

[0110] The conflict entity pair determination unit is used to enumerate and verify all matching entities within the entity bucket using the rule set for any entity bucket, and obtain the conflicting matching entity pairs corresponding to the entity bucket.

[0111] The entity bucket traversal unit is used to traverse all entity buckets and obtain all conflicting matching entity pairs.

[0112] Optionally, the enumeration task is performed using a GPU, and the conflict entity pair determination unit includes:

[0113] The thread block allocation subunit is used to allocate a thread block in the GPU for each binary star pattern rule in the rule set. The thread block is used to execute the enumeration task of the corresponding binary star pattern rule.

[0114] The Single Instruction Multiple Data (SID) subunit is used to perform pairwise verification on all matching entities within the entity bucket for any thread block in the form of SID multiple data, in order to detect whether the dependencies specified by the corresponding binary star pattern rule are satisfied.

[0115] If a conflicting entity pair determines a sub-unit and does not satisfy the dependency specified by the corresponding binary star pattern rule, then the corresponding entity pair is determined to be a conflicting matching entity pair.

[0116] Optionally, the optimized control device for single-machine image data cleaning, which uses a CPU to perform single-satellite beam matching tasks, also includes:

[0117] The buffer management module is used to place the matching entities in the N entity buckets into the buffer of the CPU after the matching entities are bucketed according to the attributes to be materialized and N entity buckets are obtained. If the buffer manager of the buffer detects that the matching entities exceed the memory capacity of the buffer, the excess part is replaced into the swap area of ​​the storage area storing the target graph data.

[0118] The GPU proxy module is used to submit the matching entities of each entity bucket in the buffer to the GPU through the GPU proxy.

[0119] Optionally, the optimized control device for single-machine map data cleaning also includes:

[0120] The conflict entity feedback module is used to store all conflicting matching entities into the CPU through the GPU agent after traversing all entity buckets and obtaining all conflicting matching entity pairs.

[0121] The graph data cleaning module 84 includes:

[0122] The graph data cleaning unit is used to use a result aggregator to perform conflict repair on conflicting matching entities in the CPU and obtain graph data cleaning results.

[0123] The cleaning result storage unit is used to save the image data cleaning result to the storage area that stores the target image data.

[0124] Optionally, the optimized control device for single-machine map data cleaning also includes:

[0125] The GPU monitoring module is used to monitor the GPU task queue and obtain the real-time GPU task queue length.

[0126] The first combination control module is used to enable the CPU to combine all single-star components that meet the combination requirements when the length of the real-time GPU task queue is less than or equal to a first threshold.

[0127] The second combination control module is used to enable the CPU to combine all single-star components that meet the combination requirements and whose common substructure is larger than the preset structure size when the length of the real-time GPU task queue is less than or equal to the second threshold and greater than the first threshold.

[0128] The third combined control module is used to prevent the CPU from performing the intersection and union operations of single-star components that include the same nodes and edges in the component structure when the length of the real-time GPU task queue is greater than the second threshold, so as to obtain a single-star bundle including a common substructure and a complete structure, and directly output the single-star component to be processed as a single-star bundle.

[0129] Optionally, the data acquisition module 81 includes:

[0130] The graph data acquisition unit is used by the CPU to acquire the target graph data from the storage area storing the target graph data;

[0131] The rule discovery unit is used to discover rules on the target graph data, obtain at least one binary star pattern rule, and form a rule set by combining all the binary star pattern rules.

[0132] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0133] In one embodiment, a computer device, which may be a server, is provided, and its internal structure diagram is shown in Figure 8. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a readable storage medium, and the database. The internal memory provides an environment for the operation of the operating system and the readable storage medium. The database stores raw user data. The network interface communicates with external terminals via a network connection. When executed by the processor, the readable storage medium implements an optimized control method for standalone image data cleaning.

[0134] In one embodiment, a computer device is provided, including a memory, a processor, and a readable storage medium stored on the memory and executable on the processor. When the processor executes the readable storage medium, it implements the steps of the optimized control method for single-machine graph data cleaning in the above embodiments, such as steps S201-S204 shown in FIG2, or the steps shown in FIG3 to 7. To avoid repetition, these steps will not be described again here. Alternatively, when the processor executes the readable storage medium, it implements the functions of each module / unit in this embodiment of the user data processing device, such as the functions of the data acquisition module 81, pattern decomposition module 82, pattern combination module 83, and graph data cleaning module 84 shown in FIG8. To avoid repetition, these functions will not be described again here.

[0135] In one embodiment, one or more readable storage media storing computer-readable instructions are provided. When executed by one or more processors, these computer-readable instructions cause the processors to perform the steps of the optimized control method for single-machine graph data cleaning described in the above embodiments, such as steps S201-S204 shown in FIG2, or the steps shown in FIG3 to 7. To avoid repetition, these steps will not be described again here. Alternatively, when the processor executes the readable storage medium, it performs the functions of each module / unit in this embodiment of the user data processing device, such as the functions of the data acquisition module 81, pattern decomposition module 82, pattern combination module 83, and graph data cleaning module 84 shown in FIG8. To avoid repetition, these functions will not be described again here. The readable storage medium in this embodiment includes non-volatile readable storage media and volatile readable storage media.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by instructing related hardware through a readable storage medium. The readable storage medium can be stored in a non-volatile computer-readable storage medium, which, when executed, can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0138] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An optimized control method for single-machine graph data cleaning, wherein, include: A set of rules is formed by acquiring target graph data and binary star pattern rules obtained by rule discovery based on the target graph data. The binary star pattern rules represent the association and dependency relationship between any two different entities. All binary star pattern rules in the rule set are decomposed into single-star components. The component structure of each single-star component is determined. Each single-star component includes a predicate condition that an entity must satisfy and an attribute to be materialized. The component structure includes at least one node and an edge connecting two nodes. The single-star components with the same nodes and edges are subjected to intersection and union operations to obtain a single-star bundle including a common substructure and a complete structure. The common substructure includes shared node connections and shared predicate conditions, and the complete structure includes all node connections and all materialized attributes of all merged single-star components. Based on each single star beam, the corresponding matching entity is matched from the target graph data. The rule set is used to enumerate and verify all matching entities to obtain conflicting matching entity pairs. The conflicting matching entity pairs are then repaired to obtain the graph data cleaning result.

2. The optimized control method for single-machine graph data cleaning according to claim 1, wherein, After matching the corresponding entity from the target map data based on each individual star beam, the process further includes: Based on the attribute to be materialized, all matching entities are bucketed to obtain N entity buckets. Matching entities with the same attribute to be materialized are in the same entity bucket, where N is a positive integer. The step of enumerating and verifying all matching entities using the rule set to obtain conflicting matching entity pairs includes: For any entity bucket, the rule set is used to enumerate and verify all matching entities within the entity bucket to obtain the conflicting matching entity pairs corresponding to the entity bucket. Iterate through all entity buckets to obtain all conflicting matching entity pairs.

3. The optimized control method for single-machine graph data cleaning according to claim 2, wherein, The GPU is used to perform an enumeration task, in which the rule set is used to enumerate and verify all matching entities within the entity bucket to obtain conflicting matching entity pairs corresponding to the entity bucket, including: In the GPU, a thread block is allocated for each binary star pattern rule in the rule set. The thread block is used to execute the enumeration task of the corresponding binary star pattern rule. For any thread block, in the form of single instruction multiple data, all matching entities in the entity bucket are checked pairwise to detect whether the dependencies specified by the corresponding binary star pattern rule are met. If the dependencies specified by the corresponding binary star pattern rule are not met, then the corresponding entity pair is determined to be a conflicting matching entity pair.

4. The optimized control method for single-machine graph data cleaning according to claim 3, wherein, After performing a single-satellite beam matching task using a CPU, and dividing all matching entities into N entity buckets based on the attributes to be materialized, the process further includes: The matching entities in the N entity buckets are placed in the buffer of the CPU. If the buffer manager detects that the matching entities exceed the memory capacity of the buffer, the excess part is swapped into the swap area of ​​the storage area storing the target graph data. The matching entities for each entity bucket in the buffer are submitted to the GPU via a GPU agent.

5. The optimized control method for single-machine graph data cleaning according to claim 4, wherein, After traversing all entity buckets and obtaining all conflicting matching entity pairs, the process also includes: The GPU agent stores all conflicting matching entities into the CPU. The process of conflict repair on the conflicting matched entity pairs to obtain graph data cleaning results includes: The result aggregator is used to perform conflict repair on the conflicting matching entities in the CPU to obtain the graph data cleaning result. The results of the graph data cleaning are saved to the storage area where the target graph data is stored.

6. The optimized control method for single-machine graph data cleaning according to claim 4, wherein, Also includes: Monitor the GPU task queue to obtain the real-time GPU task queue length; When the length of the real-time GPU task queue is less than or equal to the first threshold, the CPU is enabled to combine all single-star components that meet the combination requirements when performing the intersection and union operations of single-star components that include the same nodes and edges in the component structure to obtain a single-star bundle including common substructures and complete structures. When the length of the real-time GPU task queue is less than or equal to the second threshold and greater than the first threshold, the CPU is enabled to combine all single-star components that meet the combination requirements and whose common substructure is greater than the preset structure size when performing the intersection and union operations of single-star components that include the same nodes and edges in the component structure to obtain a single-star bundle including a common substructure and a complete structure. When the length of the real-time GPU task queue is greater than the second threshold, the CPU is prohibited from executing the intersection and union operations of single-star components that include the same nodes and edges in the component structure, to obtain a single-star bundle including the common substructure and the complete structure, and the single-star component to be processed is directly output as a single-star bundle.

7. The optimized control method for single-machine graph data cleaning according to claim 4, wherein, The rule set consisting of the acquired target map data and the binary star pattern rules obtained by rule discovery based on the target map data includes: The CPU retrieves the target map data from the storage area where the target map data is stored. Rule discovery is performed on the target graph data to obtain at least one binary star pattern rule, and all binary star pattern rules are combined into a rule set.

8. An optimized control device for single-machine image data cleaning, wherein, include: The data acquisition module is used to acquire target graph data and a set of rules consisting of binary star pattern rules obtained by rule discovery based on the target graph data. The binary star pattern rules represent the association and dependency relationship between any two different entities. The pattern decomposition module is used to decompose all binary star pattern rules in the rule set into single-star components, determine the component structure of each single-star component, and the single-star component includes a predicate condition that an entity must satisfy and an attribute to be materialized. The component structure includes at least one node and an edge connecting two nodes. The pattern combination module is used to perform intersection and union operations on single-star components with the same nodes and edges to obtain single-star bundles including common substructures and complete structures. The common substructures include shared node connection relationships and shared predicate conditions, and the complete structure includes all node connection relationships and all materialized attributes of all merged single-star components. The graph data cleaning module is used to match corresponding entities from the target graph data based on each single star beam, enumerate and verify all matching entities using the rule set, obtain conflicting matching entity pairs, perform conflict repair on the conflicting matching entity pairs, and obtain the graph data cleaning result.

9. A computer device, wherein, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the optimized control method for single-machine graph data cleaning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed by the processor, it implements the optimized control method for single-machine graph data cleaning as described in any one of claims 1 to 7.