Computer-implemented method, computer program, and computer system (measuring data quality of data in graph databases)

Abstract

To provide a computer implemented method, and a computer program and system, or a combination thereof.SOLUTION: There are provided methods and the like for performing the following operations. The operations include obtaining a first graph comprising first nodes representing first entities and first edges representing relationships between the first entities. The first nodes are associated with first entity attributes descriptive of the first entities, the first edges are associated with first edge attributes descriptive of the relationships. The operations also include determining a first subgraph for a certain node of the first nodes of the first graph. The first subgraph includes the certain node and one neighboring node of the certain node. The operations include determining a data quality issue regarding the certain node based, at least in part, on applying one or more applicable rules of a set of data quality rules to first entity attribute values and first edge attribute values of the first subgraph.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to the field of computer systems, and more particularly to data quality of data in graph databases. [Background technology]

[0002] Database systems, for example, use graph structures with nodes, edges, and properties / attributes for semantic queries to represent and store data. The graph associates data items in the database with a collection of nodes and edges, where the edges represent relationships between the nodes. Relationships allow data in the data store to be directly linked together and often retrieved in a single operation. [Prior art documents] [Patent documents]

[0003] Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been made in consideration of the above points, and aims to provide a system, method, and computer program for measuring the data quality of data in a graph database. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a computer-implemented method, computer program product, or computer system, or combinations thereof, that performs the following operations (not necessarily in the order listed): obtaining a first graph including first nodes representing first entities and first edges representing relationships between the first entities, the first nodes being associated with a first entity attribute that describes the first entities represented by the first nodes, and the first edges being associated with a first edge attribute that describes the relationships represented by the first edges; determining a first subgraph for a node of the first node in the first graph, the first subgraph including the first node and at least one neighbor of the first node; and determining data quality problems for the node based at least in part on applying one or more applicable rules of a set of data quality rules to first entity attribute values ​​and first edge attribute values ​​of the first subgraph, the set of data quality rules describing types of edges between graph nodes and relationships between attributes of the graph nodes.

[0006] Embodiments of the invention will now be described in more detail, by way of example, with reference to the following drawings: [Brief explanation of the drawings]

[0007] [Figure 1A] 1 is a schematic diagram of a computing system according to an embodiment of the present disclosure. [Figure 1B] FIG. 1 illustrates a simplified structure of a graph, according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart illustrating processing operations according to an embodiment of the present disclosure. [Figure 3] 10 is a flowchart illustrating operations for determining data quality rules according to an embodiment of the present disclosure. [Figure 4A] 10 is a flowchart illustrating operations for determining data quality rules according to an embodiment of the present disclosure. [Figure 4B]1 illustrates a graph according to an embodiment of the present disclosure. [Figure 5A] 1 is a flowchart illustrating operations for detecting data quality issues according to an embodiment of the present disclosure. [Figure 5B] 1 illustrates a graph according to an embodiment of the present disclosure. [Figure 6] A computing system suitable for performing one or more operations provided in embodiments of the present disclosure is described. DETAILED DESCRIPTION OF THE INVENTION

[0008] Various embodiments of the present invention will be described for purposes of illustration and are not intended to be exhaustive or limiting of the disclosed embodiments. Numerous modifications and / or variations will be apparent to those skilled in the art without departing from the spirit and scope of the described embodiments. The terms used herein have been chosen to best explain the principles, practical applications, or technical improvements over those found in the marketplace of the embodiments, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0009] According to aspects of the present disclosure, systems, methods, and computer programs for measuring data quality of data in a graph database can be provided. In particular, the disclosed systems, methods, and computer programs are provided for detecting inconsistencies (data quality issues) in relationships between nodes in a graph and properties / attributes of these nodes, and for detecting inconsistencies in fields (e.g., attributes, properties, etc.) of nodes. For example, embodiments of the present disclosure are provided for constructing a set of data transactions from a graph, which can be represented in a tabular format. The embodiments can further apply known association algorithms to the tabular data (set of transactions) to identify strong patterns that characterize relationships between properties / attributes of graph nodes and their relationships to other nodes, and can use this data to generate rules for association relationships. The embodiments can apply the identified association rules to all nodes and relationships in the graph to identify violations of these rules. Nodes and / or edges of the graph associated with identified violations of these association rules are then marked as potential data quality issues, which can be appropriately addressed.

[0010] A graph refers to a property graph in which data values ​​are stored as properties in nodes and edges. A property graph may be included in a graph database (GDB) and may be managed and processed by a graph database management system or other database system that provides a wrapper layer that converts the property graph, for example, into relational tables for storage and converts the relational tables into a property graph when read or queried. Such a graph may be, for example, a directed graph. A graph may be a collection of nodes (also referred to as vertices) and edges. A graph edge connects any two nodes in the graph. An edge may be represented, for example, by an ordered node pair (v1, v2) and is traversed from node v1 to node v2.

[0011] Nodes in the graph may represent entities, for example. An entity may refer to a user (e.g., an employee), an object, a company, another individual, or the like, or a combination thereof. An entity (and corresponding node) has one or more entity attributes and / or properties, to which values ​​are assigned. For example, a person may be an entity. A person's entity attributes may include, for example, marital status, age, gender, identity data, or the like, or a combination thereof. The attribute value representing a node is the value of the entity attribute value of the entity represented by the node. An edge between two nodes may be assigned one or more edge attribute values ​​that indicate at least the relationship between the two nodes connected by the edge. The attribute value representing an edge is the value of the edge attribute value. The relationship may include, for example, an inheritance relationship (e.g., parent and child) or an associative relationship according to a hierarchy, or both. For example, an inheritance relationship between nodes v1 and v2 may be referred to as an "is-a relationship" between v1 and v2 (e.g., "v2 is-a parent of v1"). An associative relationship between nodes v1 and v2 is sometimes referred to as a "has-a relationship" between v1 and v2 (e.g., "v2 has-a relationship with v1"), meaning that v1 is part of, a component of, or associated with v2.

[0012] A graph may represent entities and relationships in a set of one or more domains. A domain (also referred to as a domain ontology) may represent concepts and / or categories belonging to a part of the world, such as industry, biology, policy, or the like, or a combination thereof. A domain may typically model the definition of domain-specific terms. For example, a domain may refer to a healthcare-specific domain, an advertising-specific domain, a commerce-specific domain, a medical-specific domain, a biomedical-specific domain, or the like, or a combination thereof. A set of one or more domains represents an ontology. For example, a set of domains may include a family domain and a company domain, where the family domain and the company domain belong to the same ontology, which may be, for example, an employee ontology.

[0013] However, processing graphs associated with graph databases can be technically challenging because the graphs can often have thousands or even millions of nodes and edges, such as a graph storing data for a million customer records, contacts, or the like, or a combination of these, as well as person records associated with a company with hundreds of thousands of employees. This is particularly challenging when it comes to data deduplication, data communication over a network, or any other processing of the graph, or a combination thereof. For example, a master data management (MDM) system can use the graph as persistent storage, which needs to identify duplicate records and, if applicable, resolve duplicate record issues. This process can generally be a matching process, which can use deterministic techniques, such as phonetics (e.g., Soundex, NYSIIS), edit distance, nickname resolution, or the like, or a combination of these, or probabilistic matching techniques using fuzzy operators, or both. Additionally, the data associated with the graph can have a set of data quality problems. The data quality issues depend on the domain of the set of domains represented by the graph. For example, the set of data quality problems may include a subset of data quality problems in the family domain and another subset of data quality problems in other domains.

[0014] Aspects of the present disclosure can provide a measure of data quality in a graph database by using node context to identify data quality issues in data associated with a graph or multiple graphs in the graph database and limiting the identification of data quality issues to each subgraph structure. In some embodiments, this may allow graphs to be processed in a more efficient manner, using subgraphs and providing more reliable data by indicating data quality. For example, data quality issues for a node in a graph (e.g., a subgraph) can be determined by considering its neighboring nodes. Instead of just looking for inconsistencies within individual records, the fact that data is organized into graphs (where, for example, nodes contain attributes and have certain types of (relationship) edges or links to other nodes) adds an additional dimension: relationships between nodes in a graph can be as important as the properties of the nodes themselves when determining data quality issues for data in a graph database. Thus, embodiments of the present disclosure can provide for the detection of inconsistencies in relationships between nodes in a graph as well as the detection of inconsistencies in fields of nodes within a graph.

[0015] Generally, there are four distinct quality issues that do not appear in relational databases and can be identified when observing a graph database (GDB). First, some attributes may infer relationships (e.g., between nodes) or vice versa. Thus, inconsistencies may be discovered regarding attributes versus relationships. For example, node E1 may indicate a marital status attribute of unmarried (for the individual / entity represented by node E1). However, the edge (e.g., link) between node E1 and node E2 may indicate an edge relationship of "E1 isSpouseOf E2," which indicates a spousal relationship between the entity represented by node E1 and the entity represented by node E2. Thus, this would identify inconsistencies and potential data quality issues with the data associated with node E1 or the edge between node E1 and node E2.

[0016] In the case of triples, quality issues can arise when a relationship between an entity can infer a relationship between two other entities that were not originally nearby (e.g., not actually related as inferred). For example, if an edge (e.g., link) between nodes E1 and E2 indicates an edge relationship of "E1 isParentOf E2," and an edge (e.g., link) between nodes E1 and E3 indicates an edge relationship of "E1 isParentOf E3," this can infer "E2 isSiblingOf E3" and "E3 isSiblingOf E2" relationships between nodes E2 and E3, which may be incorrect.

[0017] Other data quality issues can arise when a relationship can infer other relationships with the same entity. For example, an edge (e.g., link) between node E1 and node C1 indicates an edge relationship of "E1 isEmployed C1", which can infer the relationship "E1 HasContract C1", which may not be accurate. Other data quality issues can arise when an incorrect bijection between two relationships is found. For example, "E1 isManager E2" cannot match or infer "E2 isManager E1".

[0018] In some embodiments, the detected data quality problem may be based on, for example, at least one of: at least one edge of a node in the first subgraph and at least one entity attribute of the node; and at least one edge of a node in the first subgraph and another edge of the node.

[0019] In some embodiments, the distance between a node in the first subgraph and any other first node in the first subgraph may be less than or equal to one of a set of predefined distances. The term "distance" refers to the distance between two nodes in the graph, which is the number of edges in the shortest path (also referred to as a geodesic on the graph) connecting the two nodes.

[0020] The first subgraph is, for example, the node n i 1 where i=0,...,n1-1, and n1 is the number of nodes in the first subgraph. i has a distinct value for each first node in the first subgraph. The first edge attribute may be E ij 1 where i and j are the first edge E ij 1 The first node n connected to i 1 and n j 1 For example, the first node is n01 The first node is referred to as n0. 1 and any other first node n in the first subgraph i 1 The distance between (i>0) may be less than or equal to a selected distance. The selected distance is selected from a predefined set of distances. In some embodiments, the selection may be, for example, a random selection. In other examples, the selection may be performed as part of an iterative process. The iterative process may include repeating the method for each distance in the set of distances. The selection of distances is performed until all distances in the set of distances have been selected. In other examples, the selection may be performed by prompting the user and receiving user input indicating the selected distance.

[0021] In some embodiments, the set of predefined distances may be, for example, user-defined, where a user can define distances that he or she considers appropriate for detecting data quality issues in the first graph. The set of distances may be, for example, distances used to (pre-)determine data quality rules to be used to detect data quality issues.

[0022] In some embodiments, the first subgraph includes a node n0 1 , one node n0 1 First edge E i0 1 other first nodes n connected via i 1 , and one node n0 1 or other node n i 1 At least one first node n adjacent to j 1 For example, the first node n j 1 is the first edge E j0 1 via node n0 1 or the first edge E ij 1 via the first node n i1 That is, the first subgraph can be linked to at least three first nodes n0 1 ,n i 1 ,n j 1 and the two first edges E i0 1 and E ij 1 / E j0 1 Such an embodiment allows for a compact structure of the first subgraph, enabling efficient identification of data quality issues without the need to examine larger subgraphs. This can be particularly advantageous because in many ontology domains, individual relationships can be defined by a small number of nodes. For example, in a family domain, the family context of a node can be defined by a small number of neighboring nodes representing family members.

[0023] For example, in some embodiments, the first graph represents entities and relationships of an ontology, such as an employee ontology. An ontology may include different domains. For example, the employee ontology may include a set of family domains and a company domain. The first subgraph may be determined, for example, such that it represents a single domain (e.g., family domain, company domain).

[0024] In some embodiments, detecting a data quality problem includes determining that one or more first entity attribute values ​​of a node and / or one or more first edge attribute values ​​of a first edge to which the node connects has a data quality problem. 1 The value of the first entity attribute of the first subgraph is j 1 and the value of the first entity attribute of the first subgraph, E ij 1In another example, in some embodiments, detecting data quality issues may be performed using rules that describe types of edges between nodes in the first subgraph and expected relationships between attributes of those nodes, or using rules that describe types of edges between nodes in the first subgraph and expected relationships between other edges between nodes in the first subgraph, where these rules are determined or generated using a graph different from the first graph.

[0025] In some embodiments, detecting a data quality problem may further include marking at least one of a node and a first edge connecting the node as having a data quality problem, which allows for optimal processing of the first graph. For example, a user may skip or consider nodes or edges labeled or marked as having a data quality problem.

[0026] In some embodiments, detecting data quality issues may further include processing a second graph including second nodes representing second entities and second edges representing relationships between the second entities. The second nodes are associated with second entity attributes describing the second entities represented by the second nodes. The second edges are associated with second edge attributes describing the relationships represented by the second edges. Processing the second graph may include generating one or more second subgraphs for each second edge of the second graph. Each second subgraph of the one or more second subgraphs has a distance between the second node of the second edge of the second subgraph and another second node that is less than or equal to a respective predefined maximum proximity level. The maximum proximity level may be an integer value greater than or equal to one. The maximum proximity level may be the same or different for each second subgraph of the second graph. The maximum proximity levels of the one or more second subgraphs define a set of distances, each distance equal to the maximum proximity level used to generate the second subgraph. Detecting the data quality issue may further include determining a data quality rule (e.g., an association rule, etc.) based on a relationship between the attribute of the second entity attribute and the attribute value of the second edge attribute of the formed second subgraph, and applying the data quality rule (e.g., the association rule, etc.) to the first graph to provide detection of the data quality issue.

[0027] In some embodiments, data quality rules (e.g., association rules) may be generated from subgraphs of the second graph. This enables automated performance of data quality measurements in the graph database, potentially reducing or eliminating user intervention. In some embodiments, the generated data quality rules (e.g., association rules) may be further used to automatically detect data quality issues in other graphs, such as the first graph. As an example, data quality rules (e.g., association rules, etc.) may be determined for each second subgraph of one or more second subgraphs. The data quality rules (e.g., association rules, etc.) of the second subgraph may be represented by combinations of values ​​of edges and entity attributes of the second subgraph. The data quality rules (e.g., association rules, etc.) may be provided in a predefined data structure, such as a tuple representing entities and their relationships and their properties / attributes. In some embodiments, the same data structure type is used to process the first subgraph and encode the value combinations of the first subgraph in the data structure. This allows the data structure provided as a rule to be compared with the data structure generated from the first graph.

[0028] For example, in some embodiments, determining data quality rules (e.g., association rules) may include traversing the second graph and computing a transaction for each edge in the second graph. A transaction is formed of all entity attribute values ​​of both the source node and target node of the edge, the type of the edge, and the types of all edges connecting the target node to any other neighboring nodes of the target node. In some embodiments, data mining techniques may be applied to detect strong patterns in the rules. In some embodiments, detected rules with too low confidence or too low support may be filtered out. Applying data mining techniques may include applying an association algorithm.

[0029] In some embodiments, detecting data quality issues may further include defining a maximum proximity level such that the number of nodes in the second subgraph is less than a maximum number of nodes, providing a configurable parameter that can be used to efficiently control processing of graphs according to embodiments of the present disclosure.

[0030] In some embodiments, determining the data quality rules may include generating a database entry for each second subgraph of the one or more second subgraphs. The database entry may include entity attribute values ​​and edge attribute values ​​for second nodes and second edges of each second subgraph. In some embodiments, an association algorithm may be applied to the database entries to detect association rules. Association rules may be selected from the detected association rules based on a confidence associated with each detected association rule (those having a confidence level above a defined level), such that the data quality rules include the selected association rules. Such embodiments provide a systematic and controllable method for generating quality rules. In some embodiments, selecting association rules may further include filtering out detected association rules having a confidence level below a threshold.

[0031] In some embodiments, the first graph and the second graph may represent the same domain ontology. In some embodiments, the first graph and the second graph may be different subgraphs of the same graph. In some embodiments, the first graph and the second graph may be the same graph, as described above, which can provide an efficient testing environment and validation of data quality problem detection.

[0032] In some embodiments, detecting data quality issues may further include processing a second graph including second nodes representing second entities and second edges representing relationships between the second entities. The second nodes are associated with second entity attributes that describe the second entities represented by the second nodes. The second edges are associated with second edge attributes that describe the relationships represented by the second edges. Processing the second graph may include, for each second edge of the second graph, generating a second subgraph having an identical structure to the first subgraph, determining data quality rules based on relationships between attribute values ​​of the second entity attributes and attribute values ​​of the second edge attributes in the generated second subgraph, and applying the data quality rules to the first graph to detect data quality issues.

[0033] The terms "first," "second," and the like are used herein as labels for the nouns they precede and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such.

[0034] 1A illustrates an exemplary computing system 100 according to an embodiment of the present disclosure. The computing system 100 may be configured to perform master data management, data warehousing, or both, such that the computing system 100 can enable a deduplication system, for example. The computing system 100 includes a data integration system 101 and one or more client systems or data sources 105. The client systems 105 may include computing systems (e.g., as described with reference to FIG. 6). The data integration system 101 may control access to a graph database system 103.

[0035] The client system 105 communicates with the data integration system 101 via a network connection, which may include, for example, a wireless local area network (WLAN) connection, a wide area network (WAN) connection, a local area network (LAN) connection, or the Internet, or a combination thereof.

[0036] The client system 105 may be configured to receive and / or generate a query request. For example, the client system 105 may generate and / or receive a query request associated with the graph database system 103. The query request may, for example, request identification of duplicate nodes. The client system 105 may send and / or forward the query request to the data integration system 101. The data integration system 101 may be configured to use the graph database system 103 to fetch data and, for example, compute an appropriate subset of the graph 107 of the graph database system 103 to send back to the client system 105 in response to the query request.

[0037] In another example, each client system 105 may be configured to send data records to the data integration system 101 to be stored by the graph database system 103. A data record (e.g., a record) may be a collection of related data items, such as a particular entity's name, birth date, class, or the like, or a combination thereof. A record may represent an entity, where the entity refers to a user, object, or concept, information about which is stored in the record. The terms "data record" and "record" are used interchangeably. The graph database system 103 may use a graph 107 to store records as entities with relationships, where each record may be assigned to a node or vertex of the graph 107, with properties being attribute values ​​such as name, birth date, or the like, or a combination thereof. The data integration system 101 may use the graph database system 103 to store records received from client systems 105, check for duplicate nodes in the graph 107, or detect data quality issues in the graph 107, or a combination thereof. For example, client system 105 may be configured to provide and / or create data records that may or may not have the same structure as graph 107. As an example, client system 105 may be configured to provide records in an Extensible Markup Language (XML) format, a JavaScript® Object Notation (JSON) format, or any other format that can provide an association between attributes and corresponding attribute values, or a combination thereof.

[0038] In one example, the data integration system 101 may use one or more extract-transform-load (ETL) batch processes to import data records from the client systems 105 via HyperText Transport Protocol (HTTP) communication and / or other types of data exchange. The data integration system 101 and / or the client systems 105 may be associated with a personal computer (PC), a server, a mobile computing device, or the like, or a combination thereof.

[0039] The data integration system 101 may be configured to process the graph 107 using one or more algorithms, such as algorithm 120, that implement at least a portion of the operations of the present disclosure. For example, the data integration system 101 may use algorithm 120 to process the data records of the graph 107 and discover data quality issues within the graph database. Although shown as a separate component, in some embodiments, the graph database system 103 may be included as part of the data integration system 101.

[0040] FIG. 1B is a diagram of a simplified structure of a graph 107 according to an example domain ontology. The example ontology relates to employees of a company. The graph 107 may include multiple nodes (e.g., person nodes) 130 representing employees of the company. The graph 107 may also include one or more nodes (e.g., spouse node, child1 node, child2 node, etc.) 130 representing people related to each employee and / or a node (e.g., company node) 130 representing the company. Each of the nodes 130 may include attribute values ​​for the entity attributes of the respective records. For example, a node (e.g., person node) 130 representing an employee may include attribute values ​​for entity attributes 135 such as status, age, identification data (ID), or the like, or a combination thereof. A node (e.g., spouse node) 130 representing a spouse may include attribute values ​​for entity attributes 135 such as status, age, identification data, or the like, or a combination thereof. Additionally, the nodes 130 representing the children (e.g., child1, child2, etc.) may also each include attribute values ​​of entity attributes 135, such as status, age, identification data, or the like, or a combination thereof, associated with the respective child.

[0041] Nodes 130 are linked together by multiple edges 131. Edges 131 may be associated with edge attribute values ​​137 that can describe the relationship between the respective nodes. For example, the edge 131 linking the person node and the spouse node is associated with the edge attribute value "Spouse-of," indicating that the individual represented by the person node is the spouse of the individual represented by the spouse node. Additionally, the edge 131 linking the person node and the child1 node is associated with the edge attribute value "parent-of," indicating that the individual represented by the person node is the parent of the individual represented by the child1 node. The edge 131 linking the person node and the child2 node is associated with the edge attribute value "parent-of," indicating that the individual represented by the person node is the parent of the individual represented by the child2 node. The edge 131 linking the child1 node and the child2 node is associated with the edge attribute value "Sibling-of," indicating that the individual represented by the child1 node is a sibling of the individual represented by the child2 node. The edge 131 linking the child2 and child1 nodes is associated with the edge attribute value "Sibling-of," indicating that the individual represented by the child2 node is a sibling of the individual represented by the child1 node. By traversing the graph 107, the data integration system 101 can determine which other nodes in the graph each node is related to.

[0042] Graphs are represented in drawings (e.g., FIGS. 1B, 4B, and 5B). Nodes (or vertices) of the graph are represented in the drawings by circles or ellipses, and edges are represented by lines connecting two respective nodes of the graph. Entities represented by the nodes, such as spouse, person, child1, child2, company, or the like, or combinations thereof, can be identified within the circle / ellipse depicting the node. For ease of illustration and description, only a small number of nodes and edges are shown, along with some of their associated entity and edge attributes. However, each node and each edge of the graph may be associated with one or more entity attributes and one or more edge attributes.

[0043] 2 is a flowchart illustrating operations 200 for processing a first graph GR1 (e.g., graph 107) and measuring the data quality of data in a graph database (e.g., to detect potential data quality issues) according to an example embodiment of the present disclosure. For illustrative purposes, the operations described in FIG. 2 may be implemented in a computing system such as that shown in FIG. 1A; however, embodiments are not limited to such implementations. In some embodiments, the operations of FIG. 2 may be performed by, for example, data integration system 101.

[0044] The operation begins at step 201, where a computing system (e.g., computing system 100, computing system 600, or the like or a combination thereof) connects a selected node n0 1 Based on the first graph GR1, the first subgraph GR1 sub For example, a node n0 is determined from the first node group of the first graph GR1 (e.g., graph 107). 1may be selected. In some embodiments, as an example, an input is received indicating a request to test for data quality issues related to a node (e.g., to determine the data quality of data in the graph). In some embodiments, a node may be randomly selected from the graph. In some embodiments, a node n0 1 The detection of data quality issues for may be part of an inspection process (eg, measuring data quality) performed for each first node of the first graph GR1 (eg, graph 107).

[0045] In some embodiments, the structure of the first graph reflects a predefined or predetermined reference structure. The reference structure may be determined, for example, as described with reference to Figure 3. As an example, the first subgraph may be formed from a node n0 1 and at least one node n0 1 adjacent node n i 1 (i=1...n1). That is, the first subgraph includes one node n0 1 , at least one neighbor node n i 1 and at least one edge E i0 1 Contains one node n0 1 is the first entity attribute set a0 1 …a0 N0 At least one neighboring node n i 1 Each of the first entity attributes a i 1 …a i Ni where N0 and Ni are the total number of entity attributes. i0 1 Each of these is a set of edge attributes b i0 1 …b i0 Ni0 may have

[0046] The operation proceeds to step 203, where the computing system calculates the number of nodes n0 in the first graph GR1. 1 The data quality problem may be detected or otherwise determined to be associated with a first subgraph GR1 of the first graph GR1. sub , comb1, comb2, ... combM. Each of the attribute combinations may include a combination of attribute a0, a combination of attribute a1, a combination of attribute a2, a combination of attribute a3, a combination of attribute a4, a combination of attribute a5, a combination of attribute a6, a combination of attribute a7, a combination of attribute a8, a combination of attribute a9, a combination of attribute a11, a combination of attribute a12, a combination of attribute a13, a combination of attribute a14, a combination of attribute a15, a combination of attribute a16, a combination of attribute a27, a combination of attribute a28, a combination of attribute a3, a combination of attribute a4, a combination of attribute a5, a combination of attribute a6, a combination of attribute a17, a combination of attribute a18, a combination of attribute a29, a combination of attribute a3, a combination of attribute a3, a combination of attribute a4, a combination of attribute a5, a combination of attribute a6, a combination of attribute a19, a combination of attribute a29 ... 1 …a0 N0 At least part of attribute a i 1 …a i Ni and at least part of attribute b i0 1 …b i0 Ni0 As an example, the combination comb1 includes the triplet (a0 1 ,a i 3 ,b io 2 ), and the combination comb2 is (a0 1 ,a0 4 ,a i 2 ,a i 3 ,b io 2 ), and so on.

[0047] In some embodiments, for example, data contained in a graph database (e.g., nodes of a subgraph with attributes and relationships to other nodes) for which data quality is to be measured (e.g., to determine data quality issues) may be processed by traversing the subgraph and constructing a set of table entries and / or transactions from the data contained in the subgraph, such that they may be represented in a tabular format, such as that described with respect to step 501 of FIG. 5A. As an example, an entry / transaction may be constructed or calculated for each edge and may consist of all attribute values ​​of both the source node and the target node connected by the edge, the edge attribute value (e.g., a type of relationship) for the edge, and all edge attribute values ​​(e.g., a type of relationship) of edges connecting the target node to any other neighboring nodes of the target node.

[0048] Further, each combination of attributes may be examined to determine whether the combination meets the expectations of what should be expressed in the attribute combination, e.g., as described with respect to step 503 of FIG. 5A. To facilitate such determination, in some embodiments, one or more criteria (e.g., data quality rules, association rules, etc.) may be generated and / or provided for each combination of multiple combinations. As an example, in some embodiments, an association algorithm may be applied to find strong patterns that characterize the relationships between the properties / attributes of the nodes of the graph and their relationships, and one or more criteria (e.g., data quality rules, association rules, etc.) may be generated or provided. The criteria (e.g., data quality rules, association rules, etc.) may represent the expected relationships between the types of edges (e.g., links) between nodes and the attributes of those nodes. For example, for combination comb1, two criteria (e.g., data quality rules, association rules, etc.) may be generated and / or provided for combination comb1. ref1 ,comb1 ref2Each of the combinations comb1, comb2... combM may be compared to one or more associated criteria (e.g., data quality rules, association rules, etc.) for use in determining or detecting data quality issues, for example, by identifying nodes and / or edges that violate the associated criteria (e.g., data quality rules, association rules, etc.).

[0049] If any one of the combinations comb1, comb2...combM does not match any of the corresponding criteria (e.g., data quality rules, association rules, etc.), such a result may indicate that there is a data quality problem with the data. If a combination violates (e.g., does not match) the corresponding criteria (e.g., data quality rules, association rules, etc.), the nodes and / or edges that violate the criteria (e.g., data quality rules, association rules, etc.) may be provided with an indication of a potential data quality problem (e.g., marked, etc.). For example, comb1 does not match any of the criteria (e.g., data quality rules, association rules, etc.) comb1 ref1 ,comb1 ref2 comb1 may be compared with each of the two criteria comb1 ref1 ,comb1 ref2 If comb1 is equal to (e.g. matches) one of the two criteria comb1 then there is no data quality issue. ref1 ,comb1 ref2 If the node or edge is not equal to (e.g., does not match) any one of the nodes, then this result indicates that there is a potential data quality problem for the data. The associated node and / or edge may be marked as having a potential data quality problem.

[0050] As an example, the attribute combination comb1 can have the following values ​​("single", "single", "spouse of"), which has a data quality issue because two single individuals cannot be married at the same time. ref1and comb1 ref2 may have the following values ​​("single", "single", "sibling of") and ("married", "married", "spouse of"), respectively. ref1 and comb1 ref2 A comparison with comb1 may reveal that there are data quality issues, and that the nodes and / or edges associated with attribute combination comb1 may be provided (e.g., marked) with an indication of the potential data quality issues.

[0051] In some embodiments, data quality rules may be applied to the first subgraph to detect data quality issues. The data quality rules may be obtained as described with reference to FIG.

[0052] FIG. 3 is a flowchart illustrating operations 300 for determining data quality rules according to an embodiment of the present disclosure.

[0053] The operation begins in step 301, where a computing system (e.g., computing system 100, computing system 600, or the like, or a combination thereof) obtains or may otherwise be provided with a graph. The graph obtained or provided in step 301 is referred to as a second graph GR2 for clarity of explanation. The second graph GR2 includes a second node n representing a second entity. i 2 and two chords n i 2 ,n j 2 A second edge E represents a relationship between the second entities of ij 2 The second node n i 2 may be associated with a second entity attribute that describes the second entity represented by the second node. ij 2may be associated with a second edge attribute that describes the relationship represented by said second edge.

[0054] The operation proceeds to step 303, where the computing system calculates the second edge E of the second graph GR2. ij 2 Regarding the second subgraph GR2 sub ij In some embodiments, a second subgraph GR2 may be formed. sub ij At least a part of each second subgraph of the first subgraph GR1 in FIG. sub For example, in some embodiments, the second subgraph GR2 sub ij Each second subgraph of is the first subgraph GR1 in Fig. 2. sub That is, the subgraph GR2 sub ij The structure of may be provided as a reference structure for the operations described in FIG. sub ij has a distance between the second node of the second edge of the second subgraph and other second nodes that is less than or equal to a predefined maximum proximity level. sub ij are provided with the same maximum proximity level. In some embodiments, as a different example, the second subgraph GR2 sub ij One subset of GR2 is provided with a maximum proximity level d1, while the second subgraph GR2 sub ij Another subset of GR2 may be provided at a maximum proximity level d2 different from d1. In some embodiments, as another example, a second subgraph GR2 sub ijEach second subgraph of may be provided with a different maximum proximity level. The maximum proximity levels used to determine the second subgraphs define the structure of each second subgraph. These structures may be provided as reference structures used to define the first subgraphs provided in the operations described in FIG.

[0055] The operation proceeds to step 305, where the computing system generates a second subgraph GR2. sub ij Data quality rules (e.g., association rules) may be determined based on the relationship between the attribute values ​​of the second entity attribute and the attribute values ​​of the second edge attribute. For example, in some embodiments, the data quality rules may be automatically generated, for example, by applying an association rule mining / learning process / algorithm to the attribute values ​​of the second subgraph. In some embodiments, as another example, a user may be prompted to provide the data quality rules based on the second subgraph. In such embodiments, user input indicating the data quality rules may be obtained by the computer system.

[0056] Figure 4A is a flowchart illustrating operations 400 for generating data quality rules according to an embodiment of the present disclosure. The operations of Figure 4A are described with reference to a graph 420 shown in Figure 4B.

[0057] The graph 420 has four nodes n1 2 ,n2 2 ,n3 2 ,n4 2 Contains node n1 2 ,n2 2 ,n3 2 ,n4 2 Each of the nodes has the entity attribute ID and Status value of the entity associated with the node. For example, node n1 2 represents an entity that is a person with ID=1 and Status=married. The graph 420 further includes a node n12 (e.g., source node) to node n2 2 (e.g., target node) 12 2 Edge E 12 2 is the node n1 2 The entity represented by node n2 2 The graph 420 further includes a node n1 having an edge attribute value "parent of" indicating that the node n1 is the parent of the entity represented by 2 Node n3 2 Edge E that links to 13 2 Edge E 13 2 is the node n1 2 The entity represented by node n3 2 The graph 420 further includes an edge attribute value "parent of" that describes the node n1 as the parent of the entity represented by 2 Node n4 2 Edge E that links to 14 2 Edge E 14 2 is the node n1 2 The entity represented by node n4 2 The graph 420 further includes a node n2 having an edge attribute value of "spouse of" indicating that the node n2 is the spouse of the entity represented by 2 Node n3 2 Edge E that links to 23 2 Edge E 23 2 is the node n1 2 The entity represented by node n3 2 Graph 420 further includes node n3, which has an edge attribute value "sibling of" indicating that it is a sibling of the entity represented by 2 Node n2 2 Edge E that links to 32 2 Edge E 322 is node n3 2 The entity represented by node n2 2 The entity represented by the edge attribute has an edge attribute value "sibling of" indicating that the entity represented by the edge attribute is a sibling of the entity represented by the edge attribute.

[0058] The operation begins at step 401, where a computer system (e.g., computing system 100, computing system 600, or the like, or a combination thereof) traverses a graph (e.g., graph 420) and finds five edges E 12 2 ,E 13 2 ,E 14 2 ,E 23 2 and E 32 2 The graph traversal and construction of entries / transactions may result in the creation of edge E, E, E's ... 12 2 ,E 13 2 ,E 14 2 ,E 23 2 and E 32 2 The following five entries / transactions T1 are associated with 2 ~T5 2 results.

[0059]

number

[0060] In this example, in an edge, E1 refers to the source node and E2 refers to the target node.

[0061] The operation proceeds to step 403, where the computer system applies a predefined or predetermined association algorithm to table entry T1 for use in determining association rules based at least in part on the relationships and properties / attributes. 2 ~T5 2 In some embodiments, the association algorithm may be, for example, an a priori algorithm. The association algorithm may be applied to transactions / entries to determine a set of association rules that have sufficient confidence (e.g., confidence above a predefined level). The association rules may represent expected relationships between types of edges (e.g., links) between nodes and attribute values ​​of those nodes. As an example, the resulting set of association rules may include the following three rules:

[0062]

number

[0063] Each association rule includes a condition and a consequence. For example, in the rules above, the consequence of each association rule is indicated or preceded by the symbol "=>".

[0064] 5A is a flowchart illustrating operations 500 for detecting data quality issues for data in a graph database according to an embodiment of the present disclosure. The operations of FIG. 5A are described with reference to a graph 520 shown in FIG. 5B.

[0065] The graph 520 has four nodes n1 1 ,n2 1 ,n3 1 ,n4 1Contains node n1 1 ,n2 1 ,n3 1 ,n4 1 Each of the nodes has the entity attribute ID and Status value of the entity associated with the node. For example, node n1 1 represents an entity that is a person with ID=1 and Status=single. Graph 520 further includes a node n1 1 (e.g., source node) to node n2 1 (e.g., target node) 12 1 Edge E 12 1 is the node n1 1 The entity represented by node n2 1 The graph 520 further includes a node n1, which has an edge attribute value "parent of" indicating that it is the parent of the entity represented by 1 Node n3 1 Edge E that links to 13 1 Edge E 13 1 is the node n1 1 The entity represented by node n3 1 The graph 520 further includes a node n1, which has an edge attribute value "parent of" indicating that it is the parent of the entity represented by 1 Node n4 1 Edge E that links to 14 1 Edge E 14 1 is the node n1 1 The entity represented by node n4 1 The graph 520 further includes a node n2 having an edge attribute value "spouse of" indicating that the node n2 is the spouse of the entity represented by 1 Node n3 1 Edge E that links to 23 1 Edge E 23 1 is node n21 The entity represented by node n3 1 The edge attribute value "parent of" describes that the entity represented by the edge is the parent of the entity represented by the edge attribute value "parent of."

[0066] The operation begins at step 501, where a computer system (e.g., computing system 100, computing system 600, or the like, or a combination thereof) traverses a graph (e.g., graph 520) and finds four edges E 12 1 ,E 13 1 ,E 14 1 and E 23 1 , and determine the properties of the relationship represented by each edge, and may build these into table entries and / or transactions (in a manner similar to that described in step 401). The properties of the relationship represented by each edge may include attribute values ​​of the source and target nodes connected by the edge, edge attribute values ​​for each edge, and edge attribute values ​​for any other edges linking the target node to neighboring nodes. As an example, these operations of traversing the graph and building entries / transactions may result in the edge E 12 1 ,E 13 1 ,E 14 1 and E 23 1 The following four entries / transactions T1 are associated with 1 ~T4 1 The output results in:

[0067]

number

[0068] In this example, E refers to the source node, and E2 refers to the target node that the edge connects to.

[0069] Operation proceeds to step 503, where the computer system applies association rules (e.g., data quality rules, etc.) such as those defined in step 403 above to the four table entries / transactions T1 and T2 obtained in step 501. 1 ,T2 1 ,T3 1 ,T4 1 As an example, the act of applying an association rule results in the following determination: Entry / Transaction T1 1 violates Rule 1 (E1.parentOf.E2 AND E1.parentOf.E3 => E2.isSibling.E3.). Rule 1 violates Rule 1 because the condition part of the rule (E1.parentOf.E2 AND E1.parentOf.E3) is true for entry / transaction T1. 1 However, the corollary of the rule (E2.isSibling.E3) is applicable because it is presented in (e.g., confirmed by) entry / transaction T1. 1 is not found (e.g., is not confirmed by) and thus entry / transaction T1 1 violates rule 1. Entry / transaction T3 1 violates rule 3 (E1.isSpouseOf.E2 => E1.status: married AND E2.status: married.). Rule 3 is violated because the condition part of the rule (E1.isSpouseOf.E2) is not true for entry / transaction T3. 1 However, entry / transaction T3 1 contains E1.status:single, and therefore the consequent of the rule (E1.status: married AND E2.status: married.) is included in entry / transaction T3. 1 Since there is no match (e.g., not confirmed by) in entry / transaction T3 1violates rule 3. Thus, entry / transaction T1 1 and entry / transaction T3 1 One or more of the nodes and / or links associated with the data are provided (e.g., marked) with an indication of potential data quality issues.

[0070] FIG. 6 illustrates a general-purpose computing system 600 suitable for implementing at least some of the operations described herein, in accordance with an embodiment of the present disclosure.

[0071] It will be understood that the operations and / or methods described herein may be at least partially non-interactive and automated by a computerized system, such as a server or embedded system. In some embodiments, however, the operations and / or methods described herein may be implemented, at least partially, in an interactive system. These operations and / or methods may also be implemented in software (e.g., software 612, BIOS 622, firmware, etc.), hardware (e.g., processor 605, etc.), or a combination thereof. In some embodiments, the operations and / or methods described herein may be implemented as an executable program in software that can be executed by a special-purpose or general-purpose digital computer, such as a personal computer, workstation, minicomputer, mainframe computer, or the like, or a combination thereof. As shown in FIG. 6, system 600 thus includes a general-purpose computer 601.

[0072] In some embodiments, with respect to the hardware architecture, as shown in FIG. 6 , a computer 601 includes a processor 605, a memory (e.g., main memory) 610 coupled to a memory controller 615, and one or more input and / or output (I / O) devices (or peripherals) 10, 645 communicatively coupled via a local input / output controller 635. The input / output controller 635 may be, but is not limited to, one or more buses or other wired or wireless connections, as known in the art. The input / output controller 635 may include additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communication, but these are omitted for simplicity. Furthermore, the local interface may include address, control, or data connections, or combinations thereof, to enable appropriate communication between the aforementioned components. As described herein, the I / O devices 10, 645 may generally include any generalized cryptographic or smart card known in the art.

[0073] The processor 605 is a hardware device specifically for executing software stored in the memory 610. The processor 605 may be any custom-made or commercially available processor, a central processing unit (CPU), a coprocessor among several processors associated with the computer 601, a semiconductor-based microprocessor (e.g., in the form of a microchip or chipset), or any device for executing software instructions, or a combination thereof.

[0074] The memory 610 may include any one or combination of volatile memory elements (e.g., random access memory (RAM) such as DRAM, SRAM, SDRAM, etc.), non-volatile memory elements (e.g., read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), etc.) It should be noted that the memory 610 may have a distributed architecture in which various components are located remotely but accessible by the processor 605.

[0075] The software 612 in memory 610 may include one or more separate programs, each including an ordered list of executable instructions for implementing logical functions, particularly functions involved in embodiments of the present disclosure. In the example of Figure 6, the software 612 in memory 610 includes instructions (e.g., instructions for managing a database, such as a database management system).

[0076] The software in memory 610 also typically includes a suitable operating system (OS) 611. The OS 611 essentially controls the execution of other computer programs, such as software 612, which implements the operations described herein.

[0077] The methods described herein may be in the form of a source program, an executable program (e.g., object code), a script, or other entity comprising a set of instructions to be executed (e.g., software 612), or a combination thereof. For source programs, the program must be translated to operate properly in conjunction with the OS 611 via a compiler, assembler, interpreter, or the like, or a combination thereof, which may or may not be included in memory 610. Furthermore, the methods may be written in an object-oriented programming language with classes of data and methods, or a procedural programming language with routines, subroutines, or functions, or a combination thereof.

[0078] In the illustrated embodiment, a conventional keyboard 650 and mouse 655 may be coupled to the input / output controller 635. Other output devices, such as I / O devices 645, may include input devices, such as, but not limited to, a printer, scanner, microphone, etc. Finally, the I / O devices 10, 645 may further include devices that communicate both input and output, such as, but not limited to, a network interface card or modulator / demodulator (for accessing other files, devices, systems, or networks), or radio frequency or other transceivers, telephone interfaces, bridges, routers, or the like, or combinations thereof. The I / O devices 10, 645 may also be any general-purpose cryptographic card or smart card known in the art. The system 600 may further include a display controller 625 coupled to the display 630. In some embodiments, the system 600 may further include a network interface for coupling to a network 665. Network 655 may be an IP-based network for communication between computer 601 and other external servers, clients, or the like, or a combination thereof, via a broadband connection. Network 665 transmits or receives data between computer 601 and external systems 30, which may be involved in performing some or all of the steps of the methods discussed herein. In an exemplary embodiment, network 665 may be a managed IP network administered by a service provider. Network 665 may be implemented in a wireless manner, for example, using wireless protocols and technologies such as Wi-Fi®, WiMAX®, or the like, or a combination thereof. Network 665 may also be a packet-switched network, such as a local area network, a wide area network, a metropolitan area network, the Internet network, or other similar types of network environments, or a combination thereof.The network 665 may be a fixed wireless network, a wireless local area network (W(LAN)), a wireless wide area network (WWAN), a personal area network (PAN), a virtual private network (VPN), an intranet, or other suitable network system or combination thereof, and includes devices for receiving and transmitting signals.

[0079] If computer 601 is a PC, workstation, intelligent device, or the like, or a combination thereof, the software in memory 610 may further include a Basic Input Output System (BIOS) 622. The BIOS is a set of essential software routines that initializes and tests hardware at startup, starts the OS 611, and supports the transfer of data between hardware devices. The BIOS is stored in ROM so that the BIOS executes when computer 601 is activated.

[0080] When computer 601 is running, processor 605 is configured to execute software 612 stored in memory 610, to communicate data to and from memory 610, and generally to control the operation of computer 601 in accordance with the software. The methods and OS 611 described herein are loaded by processor 605, possibly buffered within processor 605, and executed in whole or in part, but typically the latter.

[0081] When the systems and methods described herein are implemented in software 612, the methods may be stored on any computer-readable medium, such as storage 620, for use by or in connection with a computer-related system or method, as shown in Figure 6. Storage 620 may include disk storage, such as hard disk drive (HDD) storage.

[0082] The present invention may be a system, method, or computer program product, or combination thereof, at any level of technical detail possible in any integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0083] A computer-readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Illustrative enumeration of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch cards, or mechanically encoded devices such as ridge structures in grooves with recorded instructions, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0084] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0085] Computer-readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source or object code written in any combination of one or more programming languages, including object-oriented languages ​​such as Smalltalk®, C++, or the like, and procedural languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may execute as a stand-alone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, the electrical circuitry may execute the computer-readable program instructions by utilizing state information of the computer-readable program instructions to individualize the electrical circuitry, which may include, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), to perform aspects of the invention.

[0086] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0087] These computer-readable program instructions, when provided to a computer processor or other programmable data processing apparatus, cause the instructions, when executed by the computer processor or other programmable data processing apparatus, to create machine means for implementing the functions / acts identified in a block or blocks of the flowchart diagrams and / or blocks. These computer-readable program instructions may also be stored on a computer-readable storage medium capable of instructing a computer, programmable data processing apparatus or other device, or combination thereof, to function in a particular manner, such that a computer-readable storage medium having instructions stored thereon includes an article of manufacture containing instructions that implement aspects of the functions / acts identified in a block or blocks of the flowchart diagrams and / or blocks.

[0088] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer-implemented process such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement aspects of the functions / actions identified in a block or blocks of the flowchart and / or blocks.

[0089] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order depicted in the figures. For example, two blocks shown in succession may actually be completed as a single step simultaneously, substantially simultaneously, or in a partially or fully overlapping manner, or the blocks may be executed in the reverse order depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs specific functions or actions or that implements a combination of special-purpose hardware and computer instructions.

[0090] definition

[0091] The term "the present invention" should not be considered an absolute indication that the subject matter described by the term "the present invention" is covered by either the claims as filed or the claims that may ultimately issue after prosecution of a patent. The term "the present invention" is used to help the reader get a general feel for how the disclosures herein are believed to be potentially new, but this understanding implied by use of the term "the present invention" is tentative and provisional, and is subject to change throughout the course of patent prosecution as relevant information develops and as the claims are potentially amended.

[0092] For "embodiments," see the definition of "the present invention" above. A similar caution applies to the term "embodiments."

[0093] The phrases "and / or" or "A or B or both" or "A, B or C or a combination thereof" are inclusive or, e.g., A, B and / or C (A, B or C or a combination thereof) means that at least one of A or B or C is true or applicable.

[0094] "Including" means "including, but not necessarily limited to," unless expressly stated otherwise.

[0095] "Data communications" refers to any type of data communications scheme now known or later developed, including wireless communications, wired communications, and communications routes having wireless and wired portions. Data communications is not necessarily limited to (i) direct data communications, (ii) indirect data communications, or (iii) data communications in which the format, packetization status, medium, encryption status, or protocol, or any combination thereof, is fixed throughout the entire course of the data communications, or any combination thereof.

[0096] The words "receive / provide / transmit / input / output / report" should not be construed as implying (i) any degree of directness in the relationship between these objects and the subject, or (ii) the absence of any intervening element, act, or thing, or combination thereof, between these objects and the subject, unless expressly specified otherwise.

[0097] "Module / Sub-Module" means any set of hardware, firmware, or software, or combination thereof, operating cooperatively to perform some function, whether the module is (i) in a single local vicinity, (ii) widely distributed, (iii) in a single local vicinity within a majority of the software code, (iv) located within a single portion of the software code, (v) located in a single storage device, memory, or medium, (vi) mechanically connected, (vii) electrically connected, or (viii) connected by data communication.

[0098] "Computer" means any device having significant data processing and / or machine-readable instruction capability, including, but not limited to, desktop computers, mainframe computers, laptop computers, field programmable gate array (FPGA)-based devices, smartphones, personal digital assistants (PDAs), body-worn or insertable computers, embedded device-style computers, and application-specific integrated circuit-based devices. [Explanation of symbols]

[0099] 100...Computing system, 101...Data integration system, 103...Graph database system, 105...Client system, 107,420,520...Graph, 120...Algorithm, 130...Node, 131...Edge, 135...Entity attribute, 137...Edge attribute value, 600...Computing system, 601...Computer, 605...Processor, 610...Memory, 611...OS, 612...Software, 615...Memory controller, 622...BIOS, 625...Display controller, 630...Display, 635...Local input / output controller, 10,645...I / O device, 650...Keyboard, 655...Mouse, 30...External system, 665...Network

Claims

1. 1. A computer-implemented method, comprising: one or more processors: obtaining a first graph associated with a first ontology having a plurality of different domains, the first graph including first nodes each representing a first entity and first edges each representing a relationship between the first entities, the first nodes associated with first entity attribute values ​​describing the first entities represented by the first nodes, and the first edges associated with first edge attribute values ​​describing the relationships represented by the first edges; determining, for a node of the first node in the first graph, a first subgraph representing a single domain within the first ontology, the first subgraph including the node and at least one neighbor of the node; determining a data quality problem for the node based at least in part on applying one or more data quality rules of a set of data quality rules, the one or more data quality rules relating to a relationship between a first entity attribute value of a first node in the first subgraph and a first edge attribute value of a first edge in the first subgraph; A method comprising:

2. The data quality issue one or more first entity attribute values ​​of the one node; and One or more first edge attribute values ​​of a first edge connected to the one node determining that the associated The computer-implemented method of claim 1 , further comprising:

3. providing an indication that the data quality problem is associated with at least one of the node and a first edge to which the node is connected. The computer-implemented method of claim 1 or 2, further comprising:

4. 4. The computer-implemented method of claim 1, wherein a distance between the one node in the first subgraph and another first node in the first subgraph is less than or equal to one distance from a set of predefined distances.

5. 5. The computer-implemented method according to claim 1, wherein the first subgraph is formed by the one node, another first node connected to the one node via a first edge, and at least one first node adjacent to the one node or adjacent to the other first node connected to the one node.

6. obtaining a second graph including second nodes representing second entities and second edges representing relationships between second entities, the second nodes associated with second entity attributes describing the second entities represented by the second nodes and the second edges associated with second edge attributes describing the relationships represented by the second edges; generating one or more second subgraphs for each second edge of the second graph, each second subgraph of the one or more second subgraphs having a distance between a second node of the second edge and another second node of the second subgraph that is less than or equal to a predefined maximum proximity level, the maximum proximity level of the one or more second subgraphs defining the set of distances; determining the set of data quality rules based at least in part on relationships between attribute values ​​of second entity attributes and attribute values ​​of second edge attributes of the one or more generated second subgraphs; providing said set of data quality rules for use in detecting said data quality problems; The computer-implemented method of claim 4 further comprising:

7. defining a maximum proximity level such that the number of nodes in the second subgraph is less than the maximum number of nodes; The computer-implemented method of claim 6 further comprising:

8. Determining the set of data quality rules includes: generating a database entry for each second subgraph of the one or more second subgraphs, the database entry including entity attribute values ​​and edge attribute values ​​of the second nodes and second edges of each corresponding second subgraph; applying an association algorithm to said database entries to find association rules; selecting an association rule from the detected association rules based at least in part on a confidence level associated with each detected association rule, wherein the data quality rule includes the selected association rule; and 8. The computer-implemented method of claim 6 or 7, comprising:

9. Selecting the association rule comprises: Removing detected association rules that have a confidence level less than a threshold. The computer-implemented method of claim 8 further comprising:

10. The computer-implemented method of any one of claims 6 to 9, wherein the first graph and the second graph are subgraphs of the same graph.

11. obtaining a second graph including second nodes representing second entities and second edges representing relationships between second entities, wherein the second nodes are associated with second entity attributes describing the second entities represented by the second nodes and the second edges are associated with second edge attributes describing the relationships represented by the second edges; for each second edge of the second graph, generating a second subgraph having the same structure as the first subgraph; determining the set of data quality rules based at least in part on relationships between attribute values ​​of second entity attributes and attribute values ​​of second edge attributes in the generated second subgraph; providing said set of data quality rules for use in detecting said data quality problems; The computer-implemented method of any one of claims 1 to 5, comprising:

12. A computer-implemented method as described in any one of claims 1 to 11, wherein another data quality rule in the set of data quality rules relates to a relationship between a first edge attribute value of a first edge in the first subgraph and another first edge attribute value of at least one other first edge.

13. The computer-implemented method of claim 12, wherein the other data quality rule is (1) a rule that concludes that the first node E2 and the first node E3 have a predetermined associative relationship with each other if the conditions that the first node E1 and the first node E2 have a predetermined inheritance relationship and the first node E1 and the first node E3 have the same predetermined inheritance relationship are satisfied, or (2) a rule that concludes that the first node E5 has the same predetermined associative relationship as the first node E2 if the condition that the first node E4 has a predetermined associative relationship with the first node E5 is satisfied.

14. A computer program comprising: obtaining a first graph associated with a first ontology having a plurality of different domains, the first graph including first nodes each representing a first entity and first edges each representing a relationship between the first entities, the first nodes associated with first entity attribute values ​​describing the first entities represented by the first nodes, and the first edges associated with first edge attribute values ​​describing the relationships represented by the first edges; determining, for a node of the first node of the first graph, a first subgraph representing a single domain within the first ontology, the first subgraph including the node and at least one neighbor of the node; determining a data quality problem for the node based at least in part on applying one or more data quality rules of a set of data quality rules, the one or more data quality rules relating to a relationship between a first entity attribute value of a first node in the first subgraph and a first edge attribute value of a first edge in the first subgraph; A computer program for executing the above.

15. The computer program causes the computer to: providing an indication that the data quality problem is associated with at least one of the node and a first edge to which the node is connected.

15. The computer program product of claim 14, further comprising:

16. 16. The computer program product according to claim 14, wherein the first subgraph is formed by the one node, another first node connected to the one node via a first edge, and at least one first node adjacent to the one node or adjacent to the other first node connected to the one node.

17. The computer program causes the computer to: obtaining a second graph including second nodes representing second entities and second edges representing relationships between second entities, the second nodes associated with second entity attributes describing the second entities represented by the second nodes and the second edges associated with second edge attributes describing the relationships represented by the second edges; generating one or more second subgraphs for each second edge of the second graph, each second subgraph of the one or more second subgraphs having a distance between a second node of the second edge and another second node of the second subgraph that is less than or equal to a predefined maximum proximity level, the maximum proximity level of the one or more second subgraphs defining the set of distances; determining the set of data quality rules based at least in part on relationships between attribute values ​​of second entity attributes and attribute values ​​of second edge attributes of the one or more generated second subgraphs; providing said set of data quality rules for use in detecting said data quality problems; The computer program according to any one of claims 14 to 16, further comprising:

18. Determining the set of data quality rules includes: generating a database entry for each second subgraph of the one or more second subgraphs, the database entry including entity attribute values ​​and edge attribute values ​​of the second nodes and second edges of each corresponding second subgraph; applying an association algorithm to said database entries to find association rules; selecting an association rule from the detected association rules based at least in part on a confidence level associated with each detected association rule, wherein the data quality rule includes the selected association rule; and 18. The computer program product of claim 17, further comprising:

19. 1. A computer system comprising: a processor set; A computer-readable recording medium wherein the processor set executes program instructions stored on the computer-readable recording medium to: obtaining a first graph associated with a first ontology having a plurality of different domains, the first graph including first nodes each representing a first entity and first edges each representing a relationship between the first entities, the first nodes associated with first entity attribute values ​​describing the first entities represented by the first nodes, and the first edges associated with first edge attribute values ​​describing the relationships represented by the first edges; determining, for a node of the first node of the first graph, a first subgraph representing a single domain within the first ontology, the first subgraph including the node and at least one neighbor of the node; determining a data quality problem for the node based at least in part on applying one or more data quality rules of a set of data quality rules, the one or more data quality rules relating to a relationship between a first entity attribute value of a first node in the first subgraph and a first edge attribute value of a first edge in the first subgraph; 1. A computer system configured to execute

20. The processor set executes program instructions stored on the computer-readable recording medium to: providing an indication that the data quality problem is associated with at least one of the node and a first edge to which the node is connected.

20. The computer system of claim 19, configured to execute:

Images