Information matching using subgraphs
By identifying central nodes and grouping adjacent nodes in subgraphs to determine best-matched pairs using Hausdorff distance, the method addresses the inefficiencies of existing algorithms, reducing time and resources while enhancing accuracy in duplicate record identification.
Patent Information
- Application Number
- JP2023542590
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-01-13
- Filing Date
- 2022-01-11
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2042-01-11
AI Technical Summary
Existing software matching algorithms for identifying duplicate records in large data sets require excessive time and resources due to the exponential increase in comparison tasks as the number of records grows, and they do not effectively utilize the relational structure of the data.
A method that identifies central nodes in subgraphs and groups adjacent nodes by type, determining best-matched node pairs using Hausdorff distance to assess overall distance between central nodes, thereby reducing the time and resources needed for matching.
This approach reduces the time and resources required for matching while increasing the accuracy of identifying duplicate records by considering the relational structure of the data through subgraph comparison.
Smart Images

Figure 0007785084000016 
Figure 0007785084000017 
Figure 0007785084000018
Abstract
Description
[Technical Field]
[0001] The present disclosure relates generally to improved computer systems, and more particularly to methods, apparatus, systems, and computer program products for subgraph matching. [Background technology]
[0002] Businesses and other organizations have many data sources. These data sources contain records for people, organizations, suppliers, products, marketing plans, or other types of items. These records are often managed in multiple business systems that process the company's day-to-day transactions. These records are then moved or accessed by analytical systems to generate reports. These reports may include revenue by customer, revenue by product, sales trends, usage reports, or other types of reports. When generating reports in analytical systems, duplicate records can cause inaccuracies in the analysis and reporting of results. As a result, duplicate records in the data are identified and reconciled to meet reporting requirements.
[0003] Software matching algorithms have been used to identify duplicate records within or between different data sets. These matching algorithms implement, for example, deterministic matching, fuzzy probabilistic matching, and other types of matching processes. These software matching algorithms look at the relational and columnar data structures of records to determine whether duplicate records exist. As the number of records to be compared increases, the time and resource usage can increase exponentially.
[0004] It would therefore be desirable to have a method and apparatus that takes into account at least some of the problems discussed above, as well as other potential problems, such as overcoming technical challenges related to the amount of time and resources required to match large numbers of records. Summary of the Invention
[0005] According to one embodiment of the present invention, a method collates information. A computer system identifies a first central node in a first subgraph and a second central node in a second subgraph. The computer system identifies groups of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph. A group of adjacent nodes in the group of adjacent nodes has adjacent nodes of the same node type. The computer system identifies best-matched node pairs of adjacent nodes for each group of adjacent nodes to form a set of best-matched node pairs in a set of clusters, each best-matched node pair including a first adjacent node from the first subgraph and a second adjacent node from the second subgraph. The computer system determines whether the first central node and the second central node match using the first central node, the second central node, and the set of best-matched node pairs in the set of clusters.
[0006] According to another embodiment of the present invention, a method for matching information includes: a computer system assigning neighbor nodes of two central nodes of two subgraphs into groups by node type, each group including neighbor nodes from both of the two subgraphs; a computer system selecting a best-matching node pair of neighbor nodes for each group of neighbor nodes using Hausdorff distance to form a set of best-matching node pairs of neighbor nodes for the group of neighbor nodes; a best-matching node pair in the set of best-matching node pairs has a neighbor node from each of the two subgraphs; a computer system determining an overall distance between the two central nodes using the two central nodes and the set of best-matching node pairs of neighbor nodes; a set of best-matching node pairs for each of the two central nodes being considered for the overall distance between the two central nodes; a computer system determining whether a match exists between the two central nodes based on the overall distance between the two central nodes.
[0007] According to yet another embodiment of the present invention, an information management system includes a computer system that executes program instructions to identify a first central node in a first subgraph and a second central node in a second subgraph. The computer system executes program instructions to identify groups of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph. The groups of adjacent nodes in the group of adjacent nodes have adjacent nodes of the same node type. The computer system executes program instructions to identify best-matching node pairs of adjacent nodes for each group of adjacent nodes to form a set of best-matching node pairs. Each best-matching node pair includes a first adjacent node from the first subgraph and a second adjacent node from the second subgraph. The computer system executes program instructions to determine whether the first central node and the second central node match using the first central node, the second central node, and the set of best-matching node pairs.
[0008] According to yet another embodiment of the present invention, an information management system includes a computer system that executes program instructions to assign neighboring nodes of two central nodes in two subgraphs into groups by node type. The groups include neighboring nodes from both of the two subgraphs. The computer system executes program instructions to select, for each group of neighboring nodes, a best-matching node pair of neighboring nodes using Hausdorff distance to form a set of best-matching node pairs of neighboring nodes for a set of clusters. The best-matching node pairs in the set of best-matching node pairs have neighboring nodes from each of the two subgraphs. The computer system executes program instructions to determine an overall distance between the two central nodes using the two central nodes and the set of best-matching node pairs of neighboring nodes. The overall distance between the two central nodes is determined by considering the set of best-matching node pairs for each of the two central nodes. The computer system executes program instructions to determine whether a match exists between the two central nodes based on the overall distance between the two central nodes.
[0009] According to yet another embodiment of the present invention, a computer program product for matching information includes a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by a computer system to cause the computer to perform a method including: identifying, by the computer system, a first central node in a first subgraph and a second central node in a second subgraph; identifying, by the computer system, groups of neighboring nodes having neighboring nodes from both the first subgraph and the second subgraph, the groups of neighboring nodes in the group of neighboring nodes having neighboring nodes of the same node type; identifying, by the computer system, best-matched node pairs of neighboring nodes for each group of neighboring nodes to form a set of best-matched node pairs for a set of clusters, the best-matched node pairs including a first neighboring node from the first subgraph and a second neighboring node from the second subgraph; and determining, by the computer system, whether the first central node and the second central node match using the first central node, the second central node, and the set of best-matched node pairs in the set of clusters.
[0010] Thus, the different illustrative embodiments may reduce the time and / or resources used in determining whether pieces of information match compared to current techniques that do not compare subgraphs. Additionally, the different illustrative embodiments may also increase the accuracy in matching pieces of information, at least in primary matching or primary-secondary matching. [Brief explanation of the drawings]
[0011] [Figure 1] 1 is a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented; [Figure 2] 2 is a set of functional abstraction layers provided by the cloud computing environment 50 of FIG. 1 according to an exemplary embodiment. [Figure 3] 1 is a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented; [Figure 4] FIG. 1 is a block diagram of an information environment in accordance with an illustrative embodiment. [Figure 5] FIG. 10 is an illustration of two subgraphs with adjacent nodes assigned to groups according to an example embodiment. [Figure 6] FIG. 1 is an illustration of a group of adjacent nodes in accordance with an example embodiment. [Figure 7] 1 is an illustration of a cluster created from a group of adjacent entities in accordance with an illustrative embodiment; [Figure 8] FIG. 1 is an illustration of adjacent pieces of information in accordance with an example embodiment. [Figure 9] 10 is a flowchart of a process for managing information in accordance with an illustrative embodiment; [Figure 10] 10 is a flowchart of a process for matching central nodes in accordance with an illustrative embodiment; [Figure 11] 10 is a flowchart of a process for identifying a group of neighboring nodes according to an example embodiment. [Figure 12] 10 is a flowchart for creating a set of clusters in accordance with an example embodiment. [Figure 13] 10 is a flowchart of a process for identifying a best-matching pair of adjacent nodes in accordance with an example embodiment. [Figure 14] 10 is a flowchart of a process for determining whether a first subgraph and a second subgraph match according to an example embodiment. [Figure 15] 10 is a flowchart of a process for determining whether a first core node and a second core node match according to an example embodiment. [Figure 16] 10 is a flowchart of a process for matching subgraphs in accordance with an illustrative embodiment; [Figure 17] 10 is a flowchart of a process for assigning neighboring nodes to groups according to an example embodiment. [Figure 18] 10 is a flowchart of a process for selecting a best-matched node pair of neighboring nodes for each cluster in accordance with an illustrative embodiment. [Figure 19] 10 is a flowchart of a process for generating a feature vector in accordance with an example embodiment; [Figure 20] 10 is a flowchart of a process for matching central nodes in accordance with an illustrative embodiment; [Figure 21] 1 is a block diagram of a data processing system in accordance with an illustrative embodiment; DETAILED DESCRIPTION OF THE INVENTION
[0012] The present invention may be a system, method, or computer program product, or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium having stored thereon computer-readable program instructions for causing a processor to carry out aspects of the present invention.
[0013] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, by way of example and not limitation, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROMs, DVDs, memory sticks, floppy disks, mechanically encoded devices having instructions recorded on punch cards or ridge-in-groove structures, and the like, and suitable combinations thereof. Computer-readable storage, as used herein, should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.
[0014] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may be comprised of copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.
[0015] Computer-readable program instructions for carrying out operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk, C++, etc., and procedural programming languages such as the "C" programming language and similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, as a standalone software package, or partially on the user's computer. Alternatively, the computer may be executed partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the computer-readable program instructions in order to carry out aspects of the present invention.
[0016] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0017] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing apparatus to create a machine, such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored in a computer-readable storage medium connectable to a computer, programmable data processing apparatus, or other device, or combination thereof, that functions in a particular way, such that the computer-readable program instructions stored therein configure one of the products including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.
[0018] Computer-readable program instructions, such as instructions to perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams on a computer, other programmable apparatus, or other device, can also be loaded into a computer, other programmable data processing apparatus, or other device to perform a series of operational steps on the computer, other programmable apparatus, or other device to produce a computer-implemented process.
[0019] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of executable implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which constitute one or more executable instructions for implementing the specified logical function(s). In some alternative embodiments, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be accomplished as a single step, executed concurrently, substantially concurrently, partially, or fully in a time-overlapping manner, or the blocks may be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.
[0020] The exemplary embodiments recognize and take into account many different considerations. For example, the exemplary embodiments recognize and take into account that current matching algorithms do not consider the relationship network of records with data represented as a graph. For example, the exemplary embodiments recognize and take into account that when comparing two records about a person, if the records have the same relationships to adjacent nodes in the graph, then the records are likely to be about the same person. The exemplary embodiments recognize and take into account that comparing subgraphs can provide a stronger indication that records are duplicates compared to determining the name similarity of the records themselves. Thus, the exemplary embodiments recognize and take into account that taking subgraph comparison into account can improve matching results in the matching process.
[0021]
[0009] Accordingly, exemplary embodiments provide a method, apparatus, system, and computer program product for matching information. In one exemplary example, a first central node in a first subgraph and a second central node in a second subgraph are identified. A computer system identifies groups of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph. A group of adjacent nodes in the group of adjacent nodes has adjacent nodes of the same node type. The computer system creates a set of clusters from each group of adjacent nodes, such that each cluster in the set of clusters has adjacent nodes from both the first subgraph and the second subgraph. The computer system identifies a best-matched node pair of adjacent nodes in each cluster in the set of clusters to form a set of best-matched node pairs for the set of clusters, where the adjacent nodes of the best-matched node pair include a first adjacent node from the first subgraph and a second adjacent node from the second subgraph. Whether the first central node and the second central node match is determined by the computer system based on the first central node, the second central node, and the overall distance between the first central node and the second central node using the best-matching node pair in the set of clusters.
[0022] As used herein, a "set" when used with reference to an item means one or more items. For example, a "set of clusters" means one or more clusters. Further, a "group of" when used with reference to an item means one or more items. For example, a "group of adjacent nodes" is one or more adjacent nodes.
[0023] FIG. 1 illustrates a diagram of a cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices used by cloud consumers (e.g., PDAs or cell phones 54A, desktop computers 54B, laptop computers 54C, or automobile computer systems 54N, or combinations thereof) can communicate. The cloud computing nodes 10 can communicate with each other. The cloud computing nodes 10 can be physically or virtually grouped (not shown) in one or more networks, such as the private, community, public, or hybrid clouds described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure, platform, or software as a service, or combinations thereof, for which cloud consumers are not required to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N illustrated in FIG. 1 are exemplary only, and that the cloud computing nodes 10 in the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network-addressable connection (e.g., using a web browser), or both.
[0024] Referring now to Figure 2, there is shown a set of functional abstraction model layers provided by the cloud computing environment 50 of Figure 1. It should be understood in advance that the components, layers, and functions shown in Figure 2 are merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:
[0025] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.
[0026] The virtualization layer 70 provides an abstraction layer from which the following virtual entities can be provided, for example: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.
[0027] By way of example, the management layer 80 may provide the following functions: Resource provisioning 81 enables dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment; Metering and pricing 82 enables cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources; By way of example, these resources may include application software licenses; Security enables identification and verification of cloud consumers and tasks, as well as protection for data and other resources; User portal 83 provides consumers and system administrators with access to the cloud computing environment; Service level management 84 enables allocation and management of cloud computing resources so that requested service levels are met; Service level agreement (SLA) planning and fulfillment 85 enables advance arrangement and procurement of anticipated future cloud computing resources required in accordance with SLAs.
[0028] The workload layer 90 provides examples of functionality available to a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and data management 96. Data management 96 provides services for managing data in the cloud computing environment 50 of FIG. 1 or in a network of physical locations that access the cloud computing environment 50 of FIG. 1.
[0029] For example, the data management 96 can be implemented as a master data management service or in a data management service that can increase at least one of uniformity, accuracy, semantic consistency, and accountability in the management of information. Such management of information by the data management 96 is useful when multiple copies of information exist. The data management 96 can maintain a single version of the truth across all copies of the information. In one illustrative example, the data management 96 can be used to manage information, such as records, located in multiple operational systems. In one illustrative example, the data management 96 can identify duplicate records. The data management 96 can also reconcile the identified duplicate records. In an illustrative example, the data management 96 can employ a reconciliation process when processing information, such as records, to identify duplicate portions of information.
[0030] 3, a pictorial diagram of a network of data processing systems is depicted in which exemplary embodiments may be implemented. Network data processing system 300 is a network of computers in which exemplary embodiments may be implemented. Network data processing system 300 includes network 302, which is the medium used to provide communications links between various devices and computers connected together within network data processing system 300. Network 302 may include connections, such as wire, wireless communication links, or fiber optic cables.
[0031] In the depicted example, server computer 304 and server computer 306 connect to network 302 along with storage device 308. Additionally, client device 310 connects to network 302. As depicted, client devices 310 include client computer 312, client computer 314, and client computer 316. Client device 310 may be, for example, a computer, a workstation, or a network computer. In the depicted example, server computer 304 provides information such as boot files, operating system images, and applications to client device 310. Additionally, client device 310 may include other types of client devices, such as a mobile phone 318, a tablet computer 320, and smart glasses 322. In this illustrative example, server computer 304, server computer 306, storage device 308, and client device 310 are network devices that connect to network 302. Network 302 is the communication medium for these network devices. Some or all of the client devices 310 may form an Internet of Things (IoT) where these physical devices can connect to the network 302 and exchange information with each other over the network 302 .
[0032] Client device 310, in this example, is a client to server computer 304. Network data processing system 300 may include additional server computers, client computers, and other devices not shown. Client device 310 connects to network 302 using at least one of a wired, fiber optic, or wireless connection.
[0033] The program code located in the network data processing system 300 may be stored on a computer-recordable storage medium and downloaded for use to a data processing system or other device. For example, the program code may be stored on a computer-recordable storage medium of the server computer 304 and downloaded to the client device 310 over the network 302 for use on the client device 310.
[0034] In the depicted example, network data processing system 300 is the Internet with network 302 representing a worldwide collection of networks and gateways that use the TCP / IP (Transmission Control Protocol / Internet Protocol) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, network data processing system 300 may also be implemented using many different types of networks. For example, network 302 can comprise at least one of the Internet, an intranet, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN). Figure 3 is intended as an example, and not as architectural limitations for different illustrative embodiments.
[0035] As used herein, "a number," when used in conjunction with a reference to an item, means one or more of the item. For example, "a number of networks of different types" is one or more networks of different types.
[0036] Furthermore, the phrase "at least one," when used in conjunction with a list of items, means that different combinations of one or more of the listed items can be used, and that only one of each item in the list may be required. In other words, "at least one" means that any combination and number of items from the list can be used, but not all items in the list are required. An item can be a specific object, thing, or category.
[0037] For example, without limitation, "at least one of item A, item B, or item C" may include item A, item A and item B, or item B. This example may also include item A, item B, and item C, or item B and item C. Of course, any combination of these items may be present. In some illustrative examples, "at least one" may be, for example, without limitation, two items A, one item B, and ten items C, four items B and seven items C, or other suitable combinations.
[0038] In this illustrative example, information manager 330 is located on server computer 304. Information manager 330 may manage copies of information in the form of records 332 located in repository 334. For example, information manager 330 may identify duplicate records 336 within records 332. In the depicted example, records 332 may be for objects selected from at least one of a person, a company, an organization, a supplier, an agency, a household, a product, a service, and other suitable types of objects.
[0039] Once a match is identified within a record 332, a reconciliation can be performed. This reconciliation may include deleting duplicate copies of the record, merging the records, or other appropriate action. In this illustrative example, the duplicate records 336 may be an exact match, or may match sufficiently to represent the same object. In other words, a 100 percent match between two records may not be required in some instances for the two records to match and be designated as duplicate records 336.
[0040] For example, two records for a person may be considered duplicate records 336 even if the name is not spelled exactly the same. For example, one record may be for "John Smith" while another record is for "Jon Smith." Other information in the records may be close enough for the records to be considered a match even if the names are not an exact match. As another example, "144 River Lane" and "144 River Ln." may be considered a match for the addresses in the records.
[0041] In this illustrative example, the comparison of records 332 can be performed by information manager 330 using subgraphs. For example, information manager 330 can identify two central nodes 338 in two subgraphs 340, where each of the two central nodes 338 is in one of the two subgraphs 340. As depicted, the two subgraphs 340 also include adjacent nodes 342. Each of the two subgraphs 340 can include a portion of the adjacent nodes 342.
[0042] In this illustrative example, each neighboring node in neighboring nodes 342 may represent a record in records 332. For example, each of the two central nodes 338 may represent the record of one individual. The neighboring nodes 342 may be records or other data structures that represent objects that are connected or linked to the two central nodes 338. The objects may be selected from at least one of friends, employers, residences, contracts, vehicles, neighbors, relatives, business associates, buildings, workplaces, or some other suitable object related to one or more of the two central nodes 338.
[0043] In this illustrative example, two subgraphs 340 are compared to determine whether a match exists between the records 332 for two central nodes 338. In this illustrative example, the identification of the two central nodes 338 can be performed by the information manager 330 using any currently available matching technique. The information of the two central nodes 338 can be compared to generate feature results 344. The features are characteristics that result from the comparison of the information of the central nodes.
[0044] For example, the information can be derived from various fields of the record, such as first name, last name, first name, business address, vehicle, phone number, ZIP code, area code, or other information that may be present in the record.
[0045] The features may be characteristic in the information comparison. For example, the features may be an exact match, a partial match, missing information, no match, or other types of features. These feature results 344 may be expressed as scores or numerical values in a vector. These feature results 344 may also be used to identify candidate records for analysis by the information manager 330. The feature results 344 may also be features based on the distance between two nodes, such as two central nodes 338.
[0046] In this example, feature results 344 can be used to determine which records in records 332 can be further processed by information manager 330. In other words, feature results 344 can be used to reduce the number of records compared in identifying duplicate records 336.
[0047] Identifying the two central nodes 338 in the two subgraphs 340 enables the information manager 330 to determine a similarity 348 of the two subgraphs 340 when determining whether the records 332 represented by the two central nodes 338 are duplicate records 336. In this illustrative example, the similarity 348 may be based on the distance between the two subgraphs 340, as described below. As a result, a score 350 may be generated using the similarity 348, or both the similarity 348 and the feature results 344, to determine whether the two central nodes 338 represent duplicate records 336.
[0048] In this illustrative example, information manager 330 may make this determination by comparing score 350 to threshold values 352. These threshold values may be upper level thresholds or may define ranges for use in comparing scores 350 to determine whether two core nodes 338 represent duplicate records 336.
[0049] Thus, the information manager 330 can increase accuracy in identifying duplicate records 336. Furthermore, this accuracy can be increased in primary matching for entities such as individuals, organizations, institutions, or other single entities. Furthermore, accuracy can be increased in secondary matching for entities such as families. Determining the similarity 348 of two central nodes 338 in two subgraphs 340 can increase accuracy for secondary matching when analyzing relationship information in the two subgraphs 340.
[0050] As depicted, the information manager 330 can use the two central nodes 338 and neighboring nodes 342 of the two subgraphs 340 to the two central nodes 338 as inputs to determine the similarity 348 of the two central nodes 338. As depicted, the information manager 330 assigns the neighboring nodes 342 to groups 354. Each group in the groups 354 represents a different node type. Each group in the groups 354 has neighboring nodes 342 from both of the two subgraphs 340. Clustering can be performed to determine clusters 356 within the groups 354. In other words, each cluster of neighboring nodes 342 is a cluster of neighboring nodes 342 of the same type.
[0051] This clustering can be performed using any suitable clustering process, for example, density-based clustering can be performed on adjacent nodes 342 in groups from the two subgraphs 340.
[0052] As depicted, each cluster in clusters 356 includes adjacent nodes 342 from both of the two subgraphs 340. In other words, each cluster includes at least one adjacent node from each of the two subgraphs 340.
[0053] The information manager 330 can identify the best-matching node pair for each cluster in the cluster 356 to form the best-matching node pair 358. This determination can be made by determining the Hausdorff distance, in which the adjacency distance between two adjacent nodes is calculated from each subgraph in the cluster. This adjacency distance can be based on comparing the adjacent nodes, the links of the compared adjacent nodes, and the indices of the compared adjacent nodes. Different distances can be used to determine an overall distance 360, which can indicate the similarity 348 between two central nodes 338. The overall distance 360 is the distance between the two central nodes 338 taking into account the adjacent nodes 342. In other words, the distance between the two central nodes 338 can change when the adjacent nodes 342 are taken into account. In this example, the adjacent nodes 342 are the best-matching node pair for the two central nodes 338. The overall distance 360 for the two central nodes 338 can be used to determine whether the records 332 for the two central nodes 338 are similar enough to be considered duplicate records 336.
[0054] With reference now to Figure 4, a block diagram of an information environment is depicted in accordance with an illustrative embodiment. In this illustrative example, information environment 400 includes components that may be implemented in hardware, such as the hardware depicted in network data processing system 300 in Figure 3.
[0055] As depicted, information environment 400 is an environment in which information 402 may be managed. In this illustrative example, managing information 402 may include coordinating information 402 located in one or more of datasets 404. These datasets may be located in one or more repositories. These repositories may include, for example, at least one of a data warehouse, a data lake, a data mart, a database, or some other suitable data storage entity.
[0056] Information 402 can take a variety of forms. For example, information 402 can take the form of records 406. A record in records 406 is a data structure used to organize information 402. For example, a record can be a collection of fields, which can be of different data types. Records 406 can be stored in a database, a table, or other suitable structure.
[0057] The information management system 408 of the information environment 400 may operate to manage the information 402. This management of the information 402 may include saving, adding, deleting, modifying, or performing other operations on the information 402. For example, the information management system 408 may find duplicate information in one or more of the data sets 404. These duplicates may then be reconciled, and actions may be performed, such as de-duplication, merging duplicate information, or other actions.
[0058] In this illustrative example, information management system 408 includes a number of different components. As depicted, information management system 408 includes a computer system 410 and an information manager 412.
[0059] The information manager 412 can be implemented in software, hardware, firmware, or a combination thereof. If software is used, the operations performed by the information manager 412 can be implemented in program code configured to run on hardware, such as a processor unit. If firmware is used, the operations performed by the information manager 412 can be implemented in program code and data and stored in persistent memory for execution on a processor unit. If hardware is employed, the hardware may include circuitry that operates to perform the operations in the information manager 412.
[0060] In illustrative examples, the hardware may take the form of at least one selected from a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware, configured to perform numerous operations. In a programmable logic device, the device may be configured to perform a number of operations. The device may be later reconfigured or permanently configured to perform a number of operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field programmable logic arrays, field programmable gate arrays, and other suitable hardware devices. Furthermore, the process may be implemented with organic components integrated with inorganic components, or may consist entirely of organic components excluding humans. For example, the process may be implemented as a circuit in an organic semiconductor.
[0061] Computer system 410 is a physical hardware system and includes one or more data processing systems. When multiple data processing systems are present in computer system 410, the data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing systems may be selected from at least one of a computer, a server computer, a tablet computer, or other suitable data processing system.
[0062] In this illustrative example, the information manager 412 of the computer system 410 identifies a first central node 414 in the first subgraph 416 and a second central node 418 in the second subgraph 420. This identification can be performed in many different ways. For example, currently available comparison algorithms used to compare pieces of information, such as the records 406, to each other can be used to identify the first central node 414 and the second central node 418 from the information 402. These comparison algorithms include, for example, approximate string matching, record linkage, or other processes. In one illustrative example, each of these central nodes can be recorded in the records 406. This initial matching process can be used by the information manager 412 to identify candidate central nodes for analysis.
[0063] Additionally, in this example, the information manager 412 identifies a first subgraph 416 and a second subgraph 420. Adjacent nodes 422 of these two subgraphs are linked to one of the first central node 414 and the second central node 418.
[0064] As depicted, the information manager 412 identifies groups 424 of neighboring nodes 422 from both the first subgraph 416 and the second subgraph 420 that have neighboring nodes 422 of the same node type 428 in a node type 430. The node type 430 is structural metadata and may include metadata of different fields for pieces of information within a node. This metadata may include field names, data types, granularity, and other information. For example, a node type may be a person, organization, agency, vendor, family household, home, vehicle, contract, insurance, warranty, service, or other suitable type of metadata.
[0065] In this illustrative example, a node is a collection of information about a node type 430. A node may be, for example, a record or other suitable piece of information 402.
[0066] When creating the groups 424, the information manager 412 can arrange the adjacent nodes 422 from each subgraph into the initial group 432 based on the node type 430 of the adjacent nodes 422. The information manager 412 can select each initial group in the initial group 432 that has adjacent nodes 422 from both the first subgraph 416 of the adjacent node 422 and the second subgraph 420 of the adjacent node 422 to form the group 424 of adjacent nodes 422 that has adjacent nodes 422 from both the first subgraph 416 and the second subgraph 420.
[0067] In this illustrative example, information manager 412 creates a set of clusters 434 from each group of neighboring nodes 422 such that each cluster in set of clusters 434 has neighboring nodes 422 from both first subgraph 416 and second subgraph 420. In creating set of clusters 434, information manager 412 may create candidate clusters 436 within each group of neighboring nodes 422 in group 424 of neighboring nodes 422. Information manager 412 may select each cluster in candidate clusters 436 that has neighboring nodes 422 from both first subgraph 416 of neighboring nodes 422 and second subgraph 420 of neighboring nodes 422 to form set of clusters 434.
[0068] In the illustrative example, information manager 412 identifies best-matched node pairs 438 of adjacent nodes 422 of each cluster in set of clusters 434 to form set 440 of best-matched node pairs in set of clusters 434. The two adjacent nodes of best-matched node pair 438 include a first adjacent node 442 in adjacent node 422 from first subgraph 416 and a second adjacent node 444 in adjacent node 422 from second subgraph 420.
[0069] In identifying best matching node pairs 438, information manager 412 may determine adjacency distances 450 for compared adjacent nodes 422 within a cluster. This comparison may be based on the compared adjacent nodes 422, the links of the compared adjacent nodes 422, and the depths of the compared adjacent nodes 422. Information manager 412 may identify best matching node pairs 438 for each cluster in set of clusters 434 as the two nodes within the cluster that have the shortest adjacency distance 452 to form a set of best matching node pairs 440 for the set of clusters 434.
[0070] As depicted in this example, the information manager 412 uses a first central node 414, a second central node 418, and a set 440 of best-matching node pairs in the set of clusters 434 to determine whether the first central node 414 and the second central node 418 match based on an overall distance 446 between the first central node 414 and the second central node 418.
[0071] Additionally, the information manager 412 can use the feature results 448 to identify candidate core nodes for analysis. If two core nodes are sufficiently close to each other, an additional step can be performed to determine the overall distance 446.
[0072] In this illustrative example, feature result 448 may include features related to a comparison of information between first central node 414 and second central node 418. Feature result 448 may also include features based on the distance between first central node 414 and second central node 418. Feature result 448 may also be an aggregate based on the sum of features obtained by comparing information between first central node 414 and second central node 418. In other words, a feature is a characteristic of interest that may be present in the information being compared.
[0073] For example, the occurrence of a feature can be determined by comparing information such as last name, first name, contract name, vehicle make, vehicle model, or other types of information between two central nodes. Features can be, for example, exact matches, partial matches, similar names, left-out names, mismatched names, number of exact words, number of similar words, number of left-out words, number of mismatched words, and other types of potentially interesting features. These types of features are comparison features. The feature results 448 can include at least one individual score for different features or an aggregate score based on all features. These scores can be in the form of a feature vector, where each element of the feature vector represents the occurrence of a particular feature. In one example, the feature results 448 can be determined using currently available comparison algorithms used to identify the first central node 414 and the second central node 418.
[0074] If the two central nodes match, the information manager 412 can perform a set of actions 454 with respect to the pieces of information 402 between the first central node 414 and the second central node 418. The set of actions 454 may include, for example, de-duplication, combining the information 402, modifying the information 402, or other suitable actions.
[0075] In one illustrative example, one or more technical solutions exist that overcome technical problems related to the amount of time and resources required to match large numbers of records. As a result, the one or more technical solutions may provide a technical effect of reducing at least one of the time or amount of resources required to process information 402 to determine whether duplicate portions of information 402 exist. In one illustrative example, one or more technical solutions exist that can compare subgraphs in a manner that provides a stronger indication of whether pieces of information, such as records represented as central nodes of the subgraphs, are duplicates, compared to determining the similarity of the records themselves. In one illustrative example, one or more technical solutions exist in which subgraph comparisons are performed to improve the accuracy of record matching results.
[0076] Computer system 410 may be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware, or a combination thereof. As a result, computer system 410 operates as a special-purpose computer system in which information manager 412 within computer system 410 can determine whether pieces of information 402 match using at least one of less time or fewer resources compared to current technology. In particular, information manager 412 transforms computer system 410 into a special-purpose computer system compared to currently available general-purpose computer systems that do not have information manager 412.
[0077] In an illustrative example, the use of information manager 412 in computer system 410 integrates processes into practical applications for managing information 402 that improve the performance of computer system 410. In other words, information manager 412 in computer system 410 is directed to a practical application of the processes integrated into information manager 412 in computer system 410 that uses subgraph analysis to determine whether a match exists between information. In this illustrative example, information manager 412 in computer system 410 can identify two center nodes and subgraphs of the two center nodes and their neighboring nodes. Information manager 412 identifies groups of neighboring nodes of the two center nodes from both subgraphs based on the node types of the neighboring nodes. In other words, each group for a particular node type includes at least one neighboring node from each of the subgraphs. One or more clusters are identified by information manager 412 for the neighboring nodes in each of the groups. In this illustrative example, each of these clusters includes at least one neighboring node from each of the two subgraphs. Information manager 412 identifies the best-matching node pairs of neighboring nodes in each cluster. This identification may be performed by determining the distance between pairs of nodes and selecting the node pair with the shortest distance as the best-matching pair in the cluster. The information manager 412 may use two central nodes and the best-matching node pair identified for the cluster to determine the overall distance between the two central nodes. The information manager 412 may determine whether a match exists between the two central nodes based on the overall distance 446 between the two central nodes. The overall distance 446 is the distance between the first central node 414 and the second central node 418, taking into account neighboring nodes 442, such as the set 440 of best-matching node pairs for the first central node 414 and the second central node 418.
[0078] In this manner, a determination is made as to whether two pieces of information, such as two records corresponding to two central nodes, match. In this manner, information manager 412 in computer system 410 provides practical applications for information matching, such that the functionality of computer system 410 is improved. For example, by matching subgraphs, information manager 412 in computer system 410 can provide increased accuracy in determining whether a match exists between two pieces of information. In an illustrative example, information manager 412 can use the overall distance 446 between two central nodes to determine whether a match exists.
[0079] The illustration of information environment 400 in FIG. 4 is not intended to imply physical or architectural limitations to how illustrative embodiments may be implemented. Other components in addition to or in place of those illustrated may be used. Also, some components may be unnecessary. Moreover, the blocks are presented to illustrate some functional components. One or more of these blocks may be combined, divided, or combined and divided into different blocks when implemented in an illustrative embodiment. For example, although dataset 404 is shown as located external to computer system 410, one or more of datasets 404 may be located within computer system 410. Furthermore, if computer system 410 includes multiple data processing systems, information manager 412 may be distributed and comprise components located within multiple data processing systems. In another example, first subgraph 416 may not include any of adjacent nodes 422, while second subgraph 420 includes all of adjacent nodes 422.
[0080] 5-7 are illustrations of subgraphs that may be processed by information manager 412 of FIG. 4. Referring now to FIG. 5, an illustration of two subgraphs in which neighboring nodes are assigned to groups is depicted, according to an exemplary embodiment. In this illustrative example, first subgraph 500 includes first center node (CN1) 502, neighboring node 504, neighboring node 506, neighboring node 508, neighboring node 510, neighboring node 512, neighboring node 514, neighboring node 516, and neighboring node 518. Second subgraph 520 includes second center node (CN2) 522, neighboring node 524, neighboring node 526, neighboring node 528, neighboring node 530, neighboring node 532, neighboring node 534, neighboring node 536, and neighboring node 538. As depicted, each of the neighboring nodes has a node type. These two subgraphs are exemplary implementations for first subgraph 416 and second subgraph 420 of FIG. 4.
[0081] Turning now to Figure 6, an illustration of a group of adjacent nodes is depicted, in accordance with an exemplary embodiment. In the illustrative example, the same reference numbers may be used in multiple figures. This reuse of reference numbers in different figures represents the same elements in the different figures.
[0082] As depicted in this figure, neighboring entities in the first subgraph 500 and the second subgraph 520 are assigned or arranged into groups based on node type, in other words, all neighboring nodes in a group are of the same node type.
[0083] As depicted in this figure, group 600 includes adjacent nodes 512, 514, and 516 from first subgraph 500 and adjacent node 534 from second subgraph 520. Group 602 includes adjacent nodes 504 and 506 from first subgraph 500 and adjacent nodes 524, 526, and 528 from second subgraph 520. Group 604 includes adjacent nodes 508 and 510 from first subgraph 500 and adjacent nodes 530 and 532 from second subgraph 520.
[0084] In this illustrative example, group 606 includes adjacent node 536 and adjacent node 538 from second subgraph 520. Group 606 does not include adjacent nodes from first subgraph 500. Group 608 includes adjacent node 518 from first subgraph 500. This group does not include adjacent nodes from second subgraph 520.
[0085] Groups are selected from groups that have neighboring nodes from both subgraphs. In this example, the groups include group 600, group 602, and group 604. Groups 606 and 608 are not included in the groups for further processing. These groups do not contain neighboring nodes from both subgraphs. As a result, comparisons regarding distance or features between different subgraphs cannot be made using these groups.
[0086] 7, an illustration of clusters created from groups of adjacent entities is depicted, according to an exemplary embodiment. In this illustrative example, a cluster is created from each group of adjacent nodes that has adjacent nodes from both subgraphs within the group. Clustering is performed to group adjacent nodes such that adjacent nodes within a cluster of adjacent nodes are more similar to each other than adjacent nodes in other clusters.
[0087] This clustering is performed using an algorithm or machine learning model that implements the clustering. L This clustering can be performed using various clustering techniques, such as spatial clustering based on the density of the noisy applications. (DBSCAN) , k-means clustering, distribution-based clustering, density-based clustering, or other types of clustering can be used.
[0088] As depicted, the clustering results in the creation of cluster 700 and cluster 702 for group 600, cluster 704, cluster 706, and cluster 708 for group 602, and cluster 710 for group 604. In this illustrative example, the clusters selected for further processing are those that contain adjacent nodes from both subgraphs. As depicted, clusters 702 and 708 are removed because these clusters only contain nodes from one of the two subgraphs. The clustering result can be one or more clusters, with each cluster holding a set of adjacent nodes of the same type from each of the subgraphs. In this example, four clusters remain, and these clusters contain adjacent nodes of the same type from each of the subgraphs.
[0089] From these clusters, a best-matched node pair can be determined. A best-matched node pair can be determined for each cluster that includes adjacent nodes from both subgraphs. The best-matched node pair within a cluster is a pair of nodes from different subgraphs that have the shortest distance. In other words, the best-matched node pair includes a first adjacent node from the first subgraph 500 and a second adjacent node from the second subgraph 520, and these two adjacent nodes have the shortest distance between them within the cluster compared to other pairs of adjacent nodes within the cluster.
[0090] For example, in cluster 700, if the distance between adjacent node 516 and adjacent node 534 is 0.1 and the distance between adjacent node 514 and adjacent node 534 is 0.6, then the best matching pair is adjacent node 516 and adjacent node 534.
[0091] As another example, in cluster 704, the best matching node pair is neighbor node 504 and neighbor node 524. These are the only two nodes in the cluster. In cluster 706, neighbor node 506 and neighbor node 526 are the best matching node pair.
[0092] In cluster 710, the distance between adjacent node 510 and adjacent node 532 is 0.2, the distance between adjacent node 510 and adjacent node 530 is 0.3, the distance between adjacent node 508 and adjacent node 532 is 0.6, and the distance between adjacent node 508 and adjacent node 530 is 0.4. In this example, the closest matching node pair for cluster 710 includes adjacent node 510 and adjacent node 532. As can be seen, distances are calculated between node pairs where each node pair includes adjacent nodes from each of the two subgraphs.
[0093] These identified minimum distances may be the Hausdorff distances that apply to different subsets of node clusters. In mathematics, the Hausdorff distance measures how far two subsets of a metric space are from each other. The Hausdorff distance is also called the Hausdorff metric. For example, the Hausdorff distance of cluster 700 may be dH = min(0.1, 0.6) = 0.1. The Hausdorff distance of cluster 704 is dH = min(0.2) = 0.2, and the Hausdorff distance of cluster 706 is dH = min(0.5) = 0.5. The Hausdorff distance of cluster 710 is dH = min(0.2, 0.3, 0.6, 0.4) = 0.2.
[0094] The resulting set of Hausdorff distances is [0.1, 0.2, 0.5, 0.2], each of which is the minimum value of the best-matching node pairs in the clusters identified for groups from the first subgraph 500 and the second subgraph 520.
[0095] In this illustrative example, the distance feature vector based on the distance to the neighboring nodes can be determined based on counting the number of distances that fall within various thresholds or ranges. For example, the distance feature vector can be determined as follows: feature vector fv(i)=[number of dHs≦0.3, number of 0.7>dHs>0.3, Number of dHs≧0.7 ]. As a result, the feature vector in this example is fv(i)=[3, 1, 0].
[0096] The comparison feature vector can be determined by comparing the information of the central nodes. For example, if the first central node 502 is [John Smith Jr.] and the second central node 522 is [Johnny Smith], a feature can be identified based on a comparison of the information of these two central nodes. The feature based on the comparison of the information can be, for example, [exact name, similar name, excluded name, mismatched name]. In this example, the comparison feature vector of the central nodes is fv(i)=[1,1,1,0]. In this specific example, the first 1 is the count of [Smith against Smith], the second 1 is the count of [Johnny against John], and the third 1 is the count of [none against Jr.].
[0097] As a result, the overall feature vector including the results of the comparison feature and the neighbor distance feature of the central nodes is fv(i)=[1,1,1,0,3,1,0]. This feature vector can be used in determining the similarity of the first subgraph 500 and the second subgraph 520, where the similarity considers the first central node 502, the second central node 522, and the best-matching node pair.
[0098] In this example, similarity may be measured by the overall distance between the first central node 502 and the second central node 522. In this particular example, with a feature vector fv and a coefficient vector cv, the distance may be calculated as follows:
[0099] TIFF0007785084000001.tif20147 where cv(i) is the coefficient vector, fv(i) is the feature vector including the comparison feature and the distance feature, max(cv) is the element with the maximum value in the coefficient vector, min(cv) is the element with the minimum value in the coefficient vector, i is the index value, and n is the number of elements in the feature vector.
[0100] In this example, this feature vector, consisting of comparison features from the comparison feature vector and distance features from the distance feature vector, may be used to determine the overall distance between the first core node 502 and the second core node 522. Additionally, weighting may be applied to different feature vectors using feature vector coefficients. These coefficients may be predetermined. The coefficients may be determined using subject matter experts or machine learning models. For example, a higher feature vector coefficient may be used for certain elements in the feature vector that should be given more weight in determining the similarity of the two core nodes.
[0101] In the example depicted in Figures 5 to 7, for a feature vector of [1, 1, 1, 0, 3, 1, 0] and a coefficient vector of [10, 7, -5, -10, 5, 2, 0.5], the overall distance between the first central node and the second central node can be calculated as follows:
[0102] TIFF0007785084000002.tif14152 This gives a more accurate distance than comparing these two central nodes without considering their neighbors in the subgraph. TIFF0007785084000003.tif16144 In this depicted example, comparing subgraphs of the central nodes improves accuracy and granularity in determining similarity between records or information of the central nodes compared to comparing only the records of the central nodes. In other words, comparison of subgraphs can be performed by determining the distance between the central nodes and adjusting the determined distance between the central nodes based on neighboring nodes in the subgraphs, where the adjusted distance is the overall distance between the two central nodes.
[0103] The illustration of two central nodes and neighbor nodes for two subgraphs in Figures 5-7 is provided for the purpose of illustrating one manner in which different operations can be performed on subgraphs in an exemplary embodiment and is not meant to limit the manner in which other exemplary embodiments can be implemented. For example, eight neighbor nodes are shown for each graph. In other exemplary embodiments, other numbers of neighbor nodes may exist. For example, 3, 25, 300, or other numbers of neighbor nodes may exist in each subgraph. A subgraph may not have the same number of neighbor nodes as another subgraph that is subsequently analyzed. As another example, neighbor nodes are shown as having only a depth of 1 from the central node. In other exemplary examples, neighbor nodes may have other depths in the subgraphs, such as 2, 3, 6, or other depths. For example, a particular neighbor node may have a depth of 2 from the central node. In other exemplary examples, a particular neighbor node may have a link to another neighbor node that is linked to the central node. In another exemplary example, the feature vector may include only distance features from the distance feature vector of the neighbor nodes.
[0104] In other illustrative examples, a feature vector can be generated directly from the comparison features and distance features without generating a comparison feature vector and a distance feature vector. In some illustrative examples, a feature vector can include distance features without comparison features. In yet another illustrative example, a feature vector can be generated from a comparison of two core nodes, where the feature vector includes both comparison features and distance features. In this example, the distance features are based on the distance calculated between the two core nodes.
[0105] 8, an illustration of neighbor node information fragment is depicted in accordance with an illustrative embodiment. In this illustrative example, table 800 illustrates information that may exist for neighbor nodes.
[0106] As depicted, table 800 includes a number of different columns, which in this example include neighbor node 516 and neighbor node 534, which are the same node type in this example.
[0107] In this illustrative example, table 800 has a number of different columns that identify neighbor information, including neighbors 802, subgraph 804, link type 806, depth 808, neighbors 810, and addresses 812.
[0108] Neighbor node 802 is an identifier of a neighbor node. In this example, the neighbor node in row 814 corresponds to neighbor node 516, and the neighbor node in row 816 corresponds to neighbor node 534.
[0109] Subgraph 804, in this example, identifies the subgraph to which the near neighbor belongs. Link type 806 is an identifier for the particular type of link connecting the neighbor node to another node. The other node can be another neighbor node or a core node. The value of link type 806 indicates what type of structural metadata is present that contains information for the relationship between the two neighbor node types. In this illustrative example, link type 806 indicates a link to the neighbor's node. Depth 808 specifies the number of links connecting the neighbor node to the core node. In this example, the depth is 1 for both neighbor nodes.
[0110] In this illustrative example, neighbors 810 is a type of bucket group. The hash value of neighbors 810 is a hash value generated by hashing the neighbor's name. Addresses 812 is a bucket of addresses of neighbors identified in neighbors 810. The hash value of addresses 812 is generated from hashing the address of each neighbor. Other examples of bucket categories include phone numbers, business addresses, vehicle models, cities, countries, or other suitable categories.
[0111] In this illustrative example, a hash can be generated for a field or attribute. For a particular category, such as a name, known or allowed variations can be taken into account to generate different actions. In this way, data entry errors can be accounted for and partial matches can be identified. This type of multiple bucket hash generation for a single attribute can be applied to data such as phone numbers, dates of birth, or other suitable information.
[0112] The depiction of table 800 is limited to certain types of data for purposes of illustrating different features in one exemplary embodiment. An exemplary embodiment may have more buckets or other information in neighboring nodes. Additionally, a bucket may include multiple categories. For example, a bucket may be a name and an area code. As another example, a bucket may be Contract, Jones, and Seattle.
[0113] Turning now to FIG. 9, a flowchart of a process for managing information is depicted in accordance with an illustrative embodiment. The process of FIG. 9 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program code executed by one or more processor units located in one or more hardware devices in one or more computer systems. This process may be implemented in data management 96 of FIG. 2. In the illustrated example, this process may be implemented in information manager 330 of network data processing system 300 of FIG. 3 and information manager 412 of computer system 410 of FIG. 4. This process may be used to manage pieces of information. In this example, the pieces of information take the form of records, although particular implementations may take other forms.
[0114] The process begins by determining records in one or more datasets that are similar enough to be center nodes for use in determining subgraph similarity between center nodes (step 900). In step 900, comparisons can be made between records to obtain feature results, such as feature results 448 in FIG. 4. The results of these comparisons can be used to identify which center nodes are close enough to each other or similar enough to warrant further processing. In other words, step 900 can be performed as an initial pass in identifying candidate center nodes from records. These comparisons do not consider neighboring nodes in the subgraph, in this example. For example, the distance between center nodes can be determined based solely on the center nodes themselves.
[0115] In step 900, identifying matches between core nodes can reduce the number of comparisons that are made, so that detailed comparisons of the subgraph for the core node with the subgraphs for all other core nodes do not need to be made.
[0116] Once two central nodes are identified as sufficiently similar for further processing, the overall confidence in concluding that the two central nodes are similar or different can be increased or decreased by comparing the similarity of the contextual and independent networks of the two central nodes. These different networks are subgraphs of the two central nodes.
[0117] The process identifies a subgraph for the identified central node (step 902). The process determines the overall similarity between the central nodes (step 904). In step 904, the process can determine the overall similarity between the central nodes by considering the central node and adjacent nodes in the subgraph for the central node. For example, comparing two central nodes for "John Smith," they may themselves be somewhat similar. If the first central node only has an employment relationship with "ABC Company, Canada" and the second central node only has a partnership relationship with "XYZ," the central nodes can be interpreted as being dissimilar. However, if the second central node has an employment relationship with "ABC Company," which may or may not be a separate node from the "ABC Company, Canada" related to the first node, then it can be concluded that the two central nodes are more similar.
[0118] The process determines whether the pair of records matches based on the overall similarity of the pair of subgraphs to the pair of records (step 906). In this illustrative example, the determination may also include an analysis of feature results determined by an initial analysis of the records to identify a central node. In step 906, the record may be the central node.
[0119] The process then performs a set of actions based on whether a match is found (step 908). The process then terminates. In step 908, the actions may include at least one of de-duplication, merging of matching records, or other suitable actions may be performed. In this manner, consistency between information in different datasets may be achieved to perform operations such as reporting, transactions, or other suitable operations that require accuracy and / or consistency of records found in one or more datasets.
[0120] Turning now to FIG. 10 , a flowchart of a process for central node matching is depicted, according to an illustrative embodiment. The process of FIG. 10 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program code executed by one or more processor units located in one or more hardware devices in one or more computer systems. This process may be implemented in data management 96 of FIG. 2 . In the depicted example, this process may be implemented in information manager 330 of network data processing system 300 of FIG. 3 or information manager 412 of computer system 410 of FIG. 4 . This step of the process may be used to implement step 908 of FIG. 9 .
[0121] The process begins by identifying a first central node in a first subgraph and a second central node in a second subgraph (step 1000). The process identifies groups of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph, where the groups of adjacent nodes within the group of adjacent nodes have adjacent nodes of the same node type (step 1002).
[0122] The process creates a set of clusters from each group of adjacent nodes such that each cluster in the set of clusters has adjacent nodes from both the first subgraph and the second subgraph (step 1004). The process identifies best-matched node pairs of adjacent nodes for each cluster in the set of clusters to form a set of best-matched node pairs in the set of clusters (step 1006). In step 1006, the adjacent nodes of the best-matched node pair include a first adjacent node from the first subgraph and a second adjacent node from the second subgraph.
[0123] Using the first central node, the second central node, and the set of best-matching node pairs in the set of clusters, the process determines whether the first central node of the first subgraph and the second central node of the second subgraph match based on the overall distance between the first central node and the second central node (step 1008). In step 1008, the overall distance is different from the distance between the two central nodes without considering adjacent nodes in the subgraphs. The process then ends.
[0124] 11, a flowchart of a process for identifying a group of neighboring nodes is depicted, according to an example embodiment. The process in this figure is an example of one implementation for step 1002 of FIG.
[0125] The process begins by placing adjacent nodes from each subgraph into an initial group based on the node type of the adjacent node (step 1100). The process selects each initial group in the initial group that has adjacent nodes from both one of the adjacent node's first subgraphs and one of the adjacent node's second subgraphs to form a group of adjacent nodes that has adjacent nodes from both the first subgraph and the second subgraph (step 1102). The process then ends.
[0126] 12, a flowchart for creating a set of clusters is depicted in accordance with an illustrative embodiment. The process in this figure is an example of one implementation for step 1004 of FIG.
[0127] The process begins by creating candidate clusters within each group of adjacent nodes in groups of adjacent nodes (step 1200). The process selects each cluster within the candidate clusters that has adjacent nodes from both the first subgraph of adjacent nodes and the second subgraph of adjacent nodes to form a set of clusters (step 1202). Processing then terminates.
[0128] 13, a flowchart of a process for identifying a best-matching pair of neighboring nodes is depicted, which is an example of one implementation for step 1006 of FIG.
[0129] The process begins by determining the adjacency distances of the compared adjacent nodes in the clusters based on the compared adjacent nodes, the links of the compared adjacent nodes, and the depths of the compared adjacent nodes (step 1300). In step 1300, the adjacency distances can be determined in many different ways. For example, breadth-first search, Dijkstra algorithm, or Bellman-Ford algorithm are examples of algorithms that can be used to determine these distances.
[0130] In this example, the neighbor distance of neighboring nodes in a cluster based on the neighboring nodes being compared, the links of the neighboring nodes being compared, and the depth of the neighboring nodes being compared is calculated using one of the following formulas:
[0131] TIFF0007785084000004.tif13144
[0132] where distance(x, y) is the distance between node x and node y in the cluster, depth(x, y) is the average depth between the first depth of node x and the second depth of node y, and const is a constant value greater than 0 and less than or equal to 1. The depth of node x is the number of links with the shortest path from that node to the core node of node x. In this example, depth(x, y) can also be the average of (1) the number of shortest links between node x and the first core node and (2) the number of shortest links between node y and the second core node.
[0133] TIFF0007785084000005.tif13153
[0134] where distance(x, y) is the distance between node x and node y in the cluster, depth(x, y) is the average depth between the first depth of node x and the second depth of node y, and const is a constant value greater than 0 and less than or equal to 1. The depth of node x is the number of links with the shortest path from that node to the core node of node x.
[0135] The process identifies the best-matching node pairs for each cluster in the set of clusters as the two nodes in the cluster that have the shortest neighbor distance to form a set of best-matching node pairs for the set of clusters (step 1302). Processing then ends.
[0136] In FIG. 14, according to an exemplary embodiment First and second subgraphs 10 depicts a flowchart of a process for determining whether the following match: The illustrated process is an example of one implementation for step 1008 of FIG.
[0137] The process begins by determining the overall distance between the first central node and the second central node using a first central node, a second central node, and a set of best-matching node pairs in a set of clusters as follows:
[0138] TIFF0007785084000006.tif14153
[0139] where distance(CenterNode1, CenterNode2) is the distance between the first center node and the second center node, dH(x, y) is the distance between the neighbor node x and the neighbor node y of the best-matching node pair, and M is the number of nodes in the group. Best Matching Node Pairis the number of node types in M (step 1400). In this illustrative example, the distance, represented by dH(x, y), is a value between 0 and 1. Also, distance(CenterNode1, CenterNode2) is a value between 0 and 1. As a result, in this illustrative example, the total distance is a value between 0 and 1. In this example, a value of 0 means that there is a perfect match between the compared data, and a value of 1 means that the compared data is completely different. In some cases, it may be that some neighboring nodes of a given node type are present in the first subgraph, and some neighboring nodes of the same node type are not present in the second subgraph. These node types that do not match between the two subgraphs are not included in M.
[0140] In this example, neighbor node x can be connected by CenterNode1, and neighbor node y can be connected to CenterNode2. This connection can be direct or indirect through an intervening node. In this example, dH(x, y) is the minimum distance that can be determined for various combinations of neighbor nodes x and y that are neighbor nodes in the cluster.
[0141] The process determines whether the first subgraph and the second subgraph match based on the calculated overall distance between the first central node and the second central node (step 1402). Processing then terminates.
[0142] 15, a flowchart of a process for determining whether a first core node and a second core node match is depicted, in accordance with an illustrative embodiment. The process in this figure is an example of one implementation for step 1008 of FIG. 10.
[0143] The process begins by determining the comparison features between the first and second core nodes for the comparison feature vectors of the first and second core nodes (step 1500). A feature is a characteristic that is of interest between the information being compared. This type of feature is a comparison feature. For example, when comparing the names of the core nodes, the features of interest in the name comparison can be [number of exact words, number of similar words, number of left-over words, number of mismatched words]. When comparing "John Smith Jr." and "Johnny Smith" using these features, the element of the comparison feature vector will have a count of 1 for the number of exact words [Smith, Smith]. The second feature, the number of similar words, is denoted by [John, Johnny]. The third feature, the number of left-over words, is denoted by [Jr., none]. The fourth feature, the number of unmatched words, is 0 because there is a match. As a result, the comparison feature vector for this example is fv=[1, 1, 1, 0].
[0144] The process determines a distance feature from the minimum distance of each cluster in the set of clusters (step 1502). In this example, the distance feature can be based on whether a particular distance is within a threshold range specified for the distance feature. For example, the distance feature can be [distance less than 0.3, distance between 0.3 and 0.7, distance greater than 0.7]. In this example, there are three distance features, and the distance feature vector indicates a count of how many nodes exist for each particular feature.
[0145] The process uses the comparison feature vector and the distance feature vector to determine the overall distance between the first core node and the second core node (step 1504). In step 1504, the comparison feature vector is determined for the core node and the distance feature vector is determined for the neighboring nodes. In step 1504, the overall distance between the two core nodes taking into account the neighboring nodes in the form of best-matching node pairs is determined as follows:
[0146] TIFF0007785084000007.tif22154
[0147] where cv(i) is the element with index i of the coefficient vector, fv(i) is the element with index i of the feature vector containing the comparison feature vector and the distance feature vector, max(cv) is the element with the maximum value of the coefficient vector, min(cv) is the element with the minimum value of the coefficient vector, i is the index value, and n is the number of elements of the feature vector. In this particular example, the feature vector fv contains both the comparison feature of the central node and the distance feature of the cluster.
[0148] The feature vector, in this example, contains elements for the comparison feature of the central node and the distance feature of the neighboring nodes. The coefficient vectors are composed of elements used to weight corresponding features in the feature vector. These coefficient vectors can be used to indicate the importance of each feature in the feature vector to the overall calculation. The coefficient vectors can be predetermined or generated using a machine learning model.
[0149] The process determines whether the overall distance is within a threshold for the first core node and the second core node to match (step 1506), after which processing ends.
[0150] Referring now to FIG. 16, a flowchart of a process for matching subgraphs is depicted in accordance with an illustrative embodiment. The process of FIG. 16 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program code executed by one or more processor units located in one or more hardware devices in one or more computer systems. This process may be implemented in data management 96 of FIG. 2. In an illustrative example, this process may be implemented in information manager 330 of network data processing system 300 of FIG. 3 and information manager 412 of computer system 410 of FIG. 4. This step of the process may be used to implement step 908 of FIG. 9.
[0151] The process begins by identifying two central nodes in two subgraphs, each of which is in one of the two subgraphs (step 1600). The process assigns neighboring nodes of the two central nodes in the two subgraphs to groups by node type, where the groups include neighboring nodes from both of the two subgraphs (step 1602). The process then classifies neighboring nodes of the same node type within a group into clusters to form a set of clusters, where a cluster in the set of clusters has at least one neighboring node from each of the two subgraphs (step 1604).
[0152] The process selects the best-matching node pairs of adjacent nodes for each cluster using the Hausdorff distance to form a set of best-matching node pairs of adjacent nodes for the set of clusters (step 1606). In this example, the best-matching node pairs in the set of best-matching node pairs have adjacent nodes from each of the two subgraphs.
[0153] The process determines the overall distance between the two central nodes using the two central nodes and the set of most-matching node pairs of their neighboring nodes (step 1608). In step 1608, the overall distance between the two central nodes takes into account the set of most-matching node pairs of the two central nodes. The process determines whether a match exists between the two central nodes based on the overall distance between the two central nodes (step 1610). The process then ends.
[0154] 17 depicts a flowchart of a process for assigning neighboring nodes to groups according to an example embodiment. The process in this figure is one implementation for step 1602 in FIG.
[0155] The process begins by placing adjacent nodes from each of the two subgraphs into initial groups based on the node type of the adjacent nodes (step 1700). The process selects each initial group in the initial group that has adjacent nodes from both of the two subgraphs to form a group (step 1702). Processing then ends.
[0156] 18, a flowchart of a process for selecting the best-matching node pair of neighboring nodes for each cluster is depicted, which is an example of one implementation for step 1604 of FIG.
[0157] The process begins by determining the adjacency distances of the compared adjacent nodes within a cluster based on the compared adjacent nodes, the links of the compared adjacent nodes, and the depths of the compared adjacent nodes (step 1800). The process then identifies the best-matching node pair for each cluster in the set of clusters as the two nodes within the cluster with the shortest adjacency distance to form a set of best-matching node pairs for the set of clusters (step 1802). The process then terminates.
[0158] Turning next to FIG. 19, a flowchart of a process for generating feature vectors is depicted in accordance with an illustrative embodiment. The process of FIG. 19 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program code executed by one or more processor units located in one or more hardware devices in one or more computer systems. This process may be implemented in data management 96 of FIG. 2. In an illustrative example, this process may be implemented in information manager 330 of network data processing system 300 of FIG. 3 and information manager 412 of computer system 410 of FIG. 4.
[0159] The process begins by determining comparison features of two core nodes (step 1900). In step 1900, features are characteristics of interest present in the information being compared between the two core nodes. Next, the process determines a comparison feature vector for the comparison features (step 1902). In step 1902, each element of the comparison feature vector identifies the number of occurrences of a particular feature.
[0160] For example, when comparing names within a central node, the noteworthy features for comparing names can be [exact name, similar name, excluded name, non-matching name]. When comparing "John Smith Jr." and "Johnny Smith," for these features, the elements of the comparison feature vector for the exact name [Smith, Smith] have a count of 1. The second feature, similar name, has [John, Johnny]. The third feature, excluded name, has for distinguishing [Jr., none]. The fourth feature, non-match, has a count of 0 since there is a match. As a result, the comparison feature vector for this example is fv=[1, 1, 1, 0].
[0161] Next, the process determines a distance feature for the identified cluster for the center node (step 1904). In step 1904, the feature is based on the shortest distance within the cluster of neighboring nodes. In other words, the feature is based on the distance determined between two neighboring nodes in the best-matching node pair. The process generates a distance feature vector from the distance features (step 1906). Each element of the distance feature vector indicates the number of occurrences for a particular feature. The feature can be a threshold or range of distance between neighboring nodes.
[0162] For example, the distance features can be [distance less than 0.3, distance between 0.3 and 0.7, distance greater than 0.7]. In this example, there are three distance features, and the distance feature vector indicates a count of how many nodes there are for each particular feature.
[0163] The process then generates a feature vector that includes the comparison features in the comparison feature vector and the distance features in the distance feature vector (step 1908). The process then ends. This feature vector can be used in one approach in determining the overall distance between the central nodes.
[0164] Turning now to FIG. 20 , a flowchart of a process for matching central nodes is depicted, according to an illustrative embodiment. The process of FIG. 20 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program code executed by one or more processor units located in one or more hardware devices in one or more computer systems. This process may be implemented in data management 96 of FIG. 2 . In an illustrative example, this process may be implemented in information manager 330 of network data processing system 300 of FIG. 3 and information manager 412 of computer system 410 of FIG. 4 . This step of the process may be used to implement step 908 of FIG. 9 .
[0165] This process is similar to the steps performed in the flowchart of Figure 10. In the example shown, creating a set of clusters is an optional step.
[0166] The process begins by identifying a first central node in a first subgraph and a second central node in a second subgraph (step 2000). The process identifies a group of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph, where the group of adjacent nodes has adjacent nodes that are the same node type (step 2002).
[0167] The process identifies best-matched node pairs of adjacent nodes in each group of adjacent nodes to form a set of best-matched node pairs in the set of clusters (step 2004). In step 2004, the neighbors of each best-matched node pair include a first neighboring node from the first subgraph and a second neighboring node from the second subgraph.
[0168] Using the first central node, the second central node, and the set of best-matching node pairs in the set of clusters, the process determines whether the first central node and the second central node match based on the overall distance between the first central node and the second central node (step 2006). The process then terminates.
[0169] The flowcharts and block diagrams in the different depicted embodiments illustrate the architecture, functionality, and instructions of some possible implementations of apparatuses and methods in example embodiments. In this regard, each block in the flowcharts or block diagrams may represent at least one of a module, a segment, a function, or a portion of an operation or step. For example, one or more blocks can be implemented by program code, hardware, or a combination of program code and hardware. If implemented in hardware, the hardware can take the form of, for example, an integrated circuit that is manufactured or configured to execute one or more instructions in the flowcharts or block diagrams. If implemented as a combination of program code and hardware, the implementation can take the form of firmware. Each block in the flowcharts or block diagrams can be implemented using a special-purpose hardware system that executes different instructions, or a combination of special-purpose hardware and program code executed by the special-purpose hardware.
[0170] In some alternative implementations of the exemplary embodiments, one or more functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. Also, in a flowchart or block diagram, other blocks may be added in addition to the illustrated blocks.
[0171] Turning now to FIG. 21 , a block diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system 2100 may be used to implement cloud computing node 10 of FIG. 1 and the hardware components of hardware and software layer 60 of FIG. 2 . Data processing system 2100 may also be used to implement server computer 304, server computer 306, and client device 310 in FIG. 3 . Data processing system 2100 may also be used to implement computer system 410 in FIG. 4 . In this illustrative example, data processing system 2100 includes a communications framework 2102 that provides communications between processor unit 2104, memory 2106, persistent storage 2108, communications unit 2110, input / output (I / O) unit 2112, and display 2114. In this example, communications framework 2102 takes the form of a bus system.
[0172] The processor unit 2104 is responsible for executing instructions of software that may be loaded into the memory 2106. The processor unit 2104 includes one or more processors. For example, the processor unit 2104 may be selected from at least one of a multi-core processor, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor. Furthermore, the processor unit 2104 may be implemented using one or more heterogeneous processor systems in which a main processor exists along with secondary processors on a single chip. As another illustrative example, the processor unit 2104 may be a symmetric multiprocessor system that includes multiple processors of the same type on a single chip.
[0173] Memory 2106 and persistent storage 2108 are examples of storage 2116. Storage is any piece of hardware that can store information, such as, but not limited to, data, functional form of program code, or other suitable information, either on a temporary basis, a permanent basis, or both on a temporary and a permanent basis. Storage 2116, in these illustrative examples, may also be referred to as a computer-readable storage device. Memory 2106, in these examples, may be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 2108 may take various forms, depending on the particular implementation.
[0174] For example, persistent storage 2108 may comprise one or more components or devices. For example, persistent storage 2108 may be a hard drive, a solid-state drive (SSD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage 2108 may also be removable. For example, a removable hard drive could be used for persistent storage 2108.
[0175] Communications unit 2110, in these illustrative examples, provides for communications with other data processing systems or devices, hi these illustrative examples, communications unit 2110 is a network interface card.
[0176] The input / output unit 2112 allows for the input and output of data with other devices that may be connectable to the data processing system 2100. For example, the input / output unit 2112 may provide a connection for user input via at least one of a keyboard, a mouse, or other suitable input device. Additionally, the input / output unit 2112 may send output to a printer. The display 2114 provides a mechanism for displaying information to a user.
[0177] Instructions for at least one of the operating system, applications, or programs are located in storage 2116, which is in communication with processor unit 2104 via communications framework 2102. The processes of the different embodiments may be performed by processor unit 2104 using computer-implemented instructions, which may be located in a memory, such as memory 2106.
[0178] These instructions are program instructions, also referred to as program code, computer usable program code, or computer readable program code, that may be read and executed by a processor in processor unit 2104. The program code in different embodiments may be implemented on different physical or computer readable storage media, such as memory 2106 or persistent storage 2108.
[0179] Program code 2118 is located in a functional form on computer readable medium 2120, which is selectively removable and can be loaded onto or transferred to data processing system 2100 for execution by processor unit 2104. Program code 2118 and computer readable medium 2120 form computer program product 2122 in these illustrative examples. In the illustrative example, computer readable medium 2120 is computer readable storage medium 2124.
[0180] Computer readable storage medium 2124 is not a medium that propagates or transmits program code 2118, but rather a physical or tangible storage device used to store program code 2118. Computer readable storage medium 2124, as used herein, should not be interpreted as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted over a wire.
[0181] Alternatively, program code 2118 may be transferred to data processing system 2100 using a computer-readable signal medium. The computer-readable signal medium may be a signal, such as a propagated data signal containing program code 2118. For example, the computer-readable signal medium may be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted over a connection, such as a wireless connection, an optical fiber cable, a coaxial cable, a wire, or any other suitable type of connection.
[0182] Additionally, as used herein, "computer-readable medium 2120" may be singular or plural. For example, program code 2118 may be located on computer-readable medium 2120 in the form of a single storage device or system. In another example, program code 2118 may be located on computer-readable medium 2120 that is distributed across multiple data processing systems. In other words, some instructions of program code 2118 may be located on one data processing system and other instructions of program code 2118 may be located on another data processing system. For example, part of program code 2118 may be located on computer-readable medium 2120 of a server computer, while another part of program code 2118 may be located on computer-readable medium 2120 located on a set of client computers.
[0183] The different components illustrated for data processing system 2100 are not intended to provide architectural limitations to the manner in which different embodiments may be implemented. In some illustrative examples, one or more of the components may be incorporated into or otherwise form part of another component. For example, memory 2106, or portions thereof, may be incorporated into processor unit 2104 in some illustrative examples. The different illustrative embodiments may be implemented in a data processing system including components in addition to or in place of the components illustrated for data processing system 2100. Other components shown in FIG. 21 may vary from the illustrated illustrative example. The different embodiments may be implemented using any hardware device or system capable of running program code 2118.
[0184]
[0009] Accordingly, exemplary embodiments provide a computer-implemented method, a computer system, and a computer program product for matching information. A first central node of a first subgraph and a second central node of a second subgraph are identified by the computer system. A group of adjacent nodes having adjacent nodes from both the first subgraph and the second subgraph is identified by the computer system. A group of adjacent nodes in the group of adjacent nodes has adjacent nodes of the same node type. A set of clusters is generated by the computer system from each group of adjacent nodes such that each cluster in the set of clusters has adjacent nodes from both the first subgraph and the second subgraph. A best-matched node pair of adjacent nodes is identified by the computer system for each cluster in the set of clusters to form a set of best-matched node pairs in the set of clusters, where the adjacent nodes of the best-matched node pair include a first adjacent node in the first subgraph and a second adjacent node in the second subgraph. Whether the first central node and the second central node match is determined by the computer system based on the overall distance between the first central node and the second central node using the first central node, the second central node, and a set of best-matching node pairs in the set of clusters.
[0185] As a result, the different illustrative examples may reduce at least one of the amount of time or resources used in determining whether pieces of information match compared to current techniques that do not compare a central node with adjacent nodes in a subgraph for the central node. Additionally, the different illustrative examples may also increase the accuracy of matching pieces of information in at least the primary matching or the primary-secondary matching.
[0186] The descriptions of different exemplary embodiments are presented for purposes of illustration and description and are not intended to be exhaustive or limiting of the disclosed embodiments. The different exemplary examples describe components that perform actions or operations. In the exemplary embodiments, the components may be configured to perform the described actions or operations. For example, the components may have a structural arrangement or design that provides the components with the ability to perform the actions or operations described in the exemplary examples performed by the components. Furthermore, to the extent that the terms "includes," "including," "has," "contains," and variations thereof are used herein, such terms are intended to be inclusive in a manner similar to the open-ended term "comprise," without excluding additional or other elements.
[0187] The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Not all embodiments include all features described in the illustrative examples. Furthermore, different exemplary embodiments may provide different features compared to other exemplary embodiments. It will be apparent to those skilled in the art that many modifications and variations are possible without departing from the scope of the disclosed embodiments. The terms used herein were selected to best explain the principles of the embodiments, practical applications or technical improvements to commercially available technology, or to enable those skilled in the art to understand the embodiments described herein.
Claims
1. 1. A method for collating information, said method comprising: Identifying, by a computer system, a first central node in the first subgraph and a second central node in the second subgraph; identifying, by the computer system, a group of neighboring nodes having neighboring nodes from both the first subgraph and the second subgraph, wherein the group of neighboring nodes within the group of neighboring nodes have the neighboring nodes of the same node type; identifying, by the computer system, best-matched node pairs of the neighboring nodes of each group of neighboring nodes to form a set of best-matched node pairs, each best-matched node pair including a first neighboring node from the first subgraph and a second neighboring node from the second subgraph; determining, by the computer system, whether the first core node and the second core node match using the first core node, the second core node, and the set of best-matching node pairs; A method for causing a computer to process the above.
2. creating, by the computer system, the set of clusters from each group of the adjacent nodes such that each cluster in the set of clusters has the adjacent nodes from both the first subgraph and the second subgraph; and identifying, by the computer system, the best-matched node pairs of the adjacent nodes in each group of the adjacent nodes to form the set of best-matched node pairs, the best-matched node pairs including the first adjacent node from the first subgraph and the second adjacent node from the second subgraph. identifying, by the computer system, the best-matched node pairs of the adjacent nodes of each cluster in the set of clusters to form the set of best-matched node pairs, each best-matched node pair including the first adjacent node from the first subgraph and the second adjacent node from the second subgraph; The method of claim 1 , comprising:
3. identifying, by the computer system, the group of neighboring nodes having neighboring nodes from both the first subgraph and the second subgraph, the group of neighboring nodes within the group of neighboring nodes having the neighboring nodes of the same node type; placing, by the computer system, adjacent nodes from each subgraph into an initial group based on the node type of the adjacent nodes; selecting, by the computer system, each initial group within the initial groups having its neighboring nodes from both one of the first subgraphs of the neighboring nodes and one of the second subgraphs of the neighboring nodes to form the group of neighboring nodes having its neighboring nodes from both the first subgraph and the second subgraph; 10. The method of claim 1, comprising:
4. creating, by the computer system, the set of clusters from each group of neighboring nodes such that each cluster in the set of clusters has neighboring nodes from both the first subgraph and the second subgraph; generating, by the computer system, candidate clusters within each group of neighboring nodes within the group of neighboring nodes; selecting, by the computer system, each cluster in the candidate clusters that has neighboring nodes from both the first subgraph of neighboring nodes and the second subgraph of neighboring nodes to form the set of clusters; 3. The method of claim 2, comprising:
5. Identifying, by the computer system, the best-matching node pair for each cluster in the set of clusters, comprises: determining, by the computer system, an adjacency distance for the compared adjacent nodes within a cluster based on the compared adjacent nodes, links of the compared adjacent nodes, and depths of the compared adjacent nodes; identifying, by the computer system, the best-matched node pairs of each cluster in the set of clusters as the two nodes in the cluster that have shortest neighbor distance to form the set of best-matched node pairs for the set of clusters; 3. The method of claim 2, comprising:
6. The neighbor distance for the neighboring nodes within the cluster based on the neighboring nodes being compared, the links of the neighboring nodes being compared, and the depth of the neighboring nodes being compared is: The following formula: where distance(x, y) is the distance between node x and node y in the cluster, depth(x, y) is the average value of the first depth of node x and the second depth of node y, and const is a constant value greater than 0 and less than or equal to 1. and, where distance(x, y) is the distance between the node x and the node y in the cluster, depth(x, y) is the average value of the first depth of the node x and the second depth of the node y, and const is the constant value greater than 0 and less than or equal to 1. The method according to claim 5, wherein the calculation is performed using either one of the following:
7. Determining, by the computer system, whether the first core node and the second core node match using the first core node, the second core node, and the set of best-matching node pairs includes: The computer system calculates an overall distance between the first central node and the second central node using the first central node, the second central node, and the set of best-matching node pairs of the set of clusters as follows: Here, distance(CenterNode 1 , CenterNode 2 ) is the distance between the first core node and the second core node, dH(x,y) is the distance between adjacent nodes x and y in the best-matched node pair, and M is the number of node types for the best-matched node pairs in the group; determining, by the computer system, whether the first core node and the second core node match based on the calculated overall distance between the first core node and the second core node; The method of claim 2 , comprising:
8. Determining, by the computer system, whether the first core node and the second core node match using the first core node, the second core node, and the set of best-matching node pairs includes: comparing, by the computer system, the first core node with the second core node to determine a comparison characteristic of the first core node with the second core node; determining, by the computer system, a distance characteristic from a minimum distance between the adjacent nodes of each cluster in the set of clusters; determining, by the computer system, an overall distance between the first core node and the second core node using the comparison feature and the distance feature; determining, by the computer system, whether the overall distance is within a threshold for the first core node and the second core node to match; The method of claim 2 , comprising:
9. The total distance between the first central node and the second central node is given as follows:
9. The method of claim 8, wherein cv(i) is a coefficient vector, fv(i) is a feature vector including the comparison feature and the distance feature, max(cv) is the maximum value among the elements of the coefficient vector, min(cv) is the minimum value among the elements of the coefficient vector, i is an index value, and n is the number of elements in the feature vector.
10. 1. A method of collating information, said method comprising: assigning, by a computer system, adjacent nodes of two central nodes of two subgraphs into groups by node type, the groups including adjacent nodes from both of the two subgraphs; selecting, by the computer system, for each group of neighboring nodes, a best-matched node pair of the neighboring nodes using Hausdorff distance to form a set of best-matched node pairs of the neighboring nodes for the group of neighboring nodes, the best-matched node pairs in the set having neighboring nodes from each of the two subgraphs; determining, by the computer system, an overall distance between the two core nodes using the set of most-matched node pairs of the two core nodes and the neighboring nodes, wherein the overall distance between the two core nodes takes into account the set of most-matched node pairs for each of the two core nodes; determining whether a match exists between the two core nodes based on the overall distance between the two core nodes; A method comprising:
11. clustering, by the computer system, adjacent nodes of the same node type in the group to form a set of clusters, each cluster in the set of clusters having at least one adjacent node from each of the two subgraphs; further comprising selecting, by the computer system, for each group of neighboring nodes, the best-matched node pairs of the neighboring nodes using the Hausdorff distance to form the set of best-matched node pairs of the neighboring nodes for the group of neighboring nodes, the best-matched node pairs in the set having neighboring nodes from each of the two subgraphs, the selecting including: selecting, by the computer system, for each cluster, the best-matched node pairs of the adjacent nodes using the Hausdorff distance to form the set of best-matched node pairs of the adjacent nodes for the set of clusters, the best-matched node pairs in the set having adjacent nodes from each of the two subgraphs; 11. The method of claim 10, comprising:
12. assigning, by a computer system, the adjacent nodes of the two central nodes of two subgraphs to the groups by the node type, the groups including the adjacent nodes from both of the two subgraphs, the assigning including: by the computer system, based on the node type of the neighboring node: placing the adjacent nodes from each of the two subgraphs into an initial group; selecting, by the computer system, each initial group in the initial group that has adjacent nodes from both of the two subgraphs to form the group; 12. The method of claim 11, comprising:
13. Identifying a first central node in the first subgraph and a second central node in the second subgraph; Identifying a group of neighboring nodes having neighboring nodes from both the first subgraph and the second subgraph, wherein the group of neighboring nodes within the group of neighboring nodes have the neighboring nodes of the same node type; identifying best-matched node pairs of the neighboring nodes of each group of the neighboring nodes to form a set of best-matched node pairs, each best-matched node pair including a first neighboring node from the first subgraph and a second neighboring node from the second subgraph; Using the first central node, the second central node, and the set of best-matching node pairs, determine whether the first central node and the second central node match. A computer system that executes program instructions Information management system including:
14. The computer system includes: and executing program instructions to create the set of clusters from each group of the adjacent nodes such that each cluster in the set of clusters has the adjacent nodes from both the first subgraph and the second subgraph, and identifying the best-matched node pairs of the adjacent nodes in each group of the adjacent nodes to form a set of best-matched node pairs, the adjacent nodes of the best-matched node pair including the first adjacent node from the first subgraph and the second adjacent node from the second subgraph; 14. The information management system of claim 13, executing program instructions to identify best-matched node pairs of the adjacent nodes of each cluster in the set of clusters to form the set of best-matched node pairs, each best-matched node pair including the first adjacent node from the first subgraph and the second adjacent node from the second subgraph.
15. In identifying the group of neighboring nodes having the neighboring nodes from both the first subgraph and the second subgraph, the group of neighboring nodes within the group of neighboring nodes have the neighboring nodes of the same node type, and the computer system: placing the adjacent nodes from each subgraph into an initial group based on the node type of the adjacent nodes; selecting each initial group in the initial groups having its neighboring nodes from both one of the first subgraphs of the neighboring nodes and one of the second subgraphs of the neighboring nodes to form the groups of neighboring nodes having its neighboring nodes from both the first subgraphs and the second subgraphs; 14. The information management system of claim 13, which executes program instructions.
16. In creating the set of clusters from each group of adjacent nodes, such that each cluster in the set of clusters has adjacent nodes from both the first subgraph and the second subgraph, the computer system: creating candidate clusters within each group of neighboring nodes within the group of neighboring nodes; selecting each cluster in the candidate clusters that has neighboring nodes from both the first subgraph of neighboring nodes and the second subgraph of neighboring nodes to form the set of clusters; 15. The information management system of claim 14, which executes program instructions.
17. In identifying the best-matching node pair for each cluster in the set of clusters, the computer system: determining a proximity distance for the compared neighboring nodes within a cluster based on the compared neighboring nodes, the links of the compared neighboring nodes, and the depths of the compared neighboring nodes; identifying the best-matched node pair for each cluster in the set of clusters as the two nodes in the cluster that have the shortest neighbor distance to form the set of best-matched node pairs for the set of clusters; 15. The information management system of claim 14, which executes program instructions.
18. The neighbor distance for the neighboring nodes within the cluster based on the neighboring nodes being compared, the links of the neighboring nodes being compared, and the depth of the neighboring nodes being compared is: The following formula: where distance(x, y) is the distance between node x and node y in the cluster, depth(x, y) is the average value of the first depth of node x and the second depth of node y, and const is a constant value greater than 0 and less than or equal to 1. and, where distance(x, y) is the distance between the node x and the node y in the cluster, depth(x, y) is the average value of the first depth of the node x and the second depth of the node y, and const is the constant value greater than 0 and less than or equal to 1.
18. The information management system according to claim 17, wherein the calculation is performed using any one of the following:
19. When determining whether the first core node and the second core node match using the first core node, the second core node, and the set of best-matching node pairs, the computer system: Using the first central node, the second central node, and the set of best-matching node pairs of the set of clusters, calculate the overall distance between the first central node and the second central node as follows: Here, distance(CenterNode 1 , CenterNode 2 ) is the distance between the first core node and the second core node, dH(x,y) is the distance between adjacent nodes x and y in the best-matched node pair, and M is the number of node types for the best-matched node pairs in the group; determining whether the first central node and the second central node match based on the calculated overall distance between the first central node and the second central node; 15. The information management system of claim 14, which executes program instructions.
20. In determining whether the first central node and the second central node match using the first central node, the second central node, and the set of best-matching node pairs of the set of clusters, the computer system: comparing the first central node with the second central node to determine a comparison characteristic of the first central node with the second central node; determining a distance feature from the minimum distance between adjacent nodes of each cluster in said set of clusters; determining an overall distance between the first central node and the second central node using the comparison feature and the distance feature; Determine whether the overall distance is within a threshold for the first core node and the second core node to match.
15. The information management system of claim 14, which executes program instructions.
21. The total distance between the first central node and the second central node is given as follows:
21. The information management system of claim 20, wherein cv(i) is a coefficient vector, fv(i) is a feature vector including the comparison feature and the distance feature, max(cv) is the maximum value among the elements of the coefficient vector, min(cv) is the minimum value among the elements of the coefficient vector, i is an index value, and n is the number of elements of the feature vector.
22. assigning adjacent nodes of two central nodes of two subgraphs to groups by node type, the groups including adjacent nodes from both of the two subgraphs; selecting, for each group of neighboring nodes, a best-matched node pair of the neighboring nodes using Hausdorff distance to form a set of best-matched node pairs of the neighboring nodes for the group of neighboring nodes, the best-matched node pairs in the set having neighboring nodes from each of the two subgraphs; determining an overall distance between the two core nodes using the set of most-matched node pairs of the two core nodes and the neighboring nodes, the overall distance between the two core nodes being determined taking into account the set of most-matched node pairs for each of the two core nodes; determining whether a match exists between the two core nodes based on the overall distance between the two core nodes; An information management system that includes a computer system that executes program instructions.
23. The computer system includes: executing the program instructions to group adjacent nodes of the same node type in the group into clusters to form a set of clusters, each cluster in the set of clusters having at least one adjacent node from each of the two subgraphs; selecting, for each group of adjacent nodes, the best-matched node pairs of the adjacent nodes using the Hausdorff distance to form the set of best-matched node pairs of the adjacent nodes for the group of adjacent nodes, the best-matched node pairs in the set of best-matched node pairs having adjacent nodes from each of the two subgraphs, and the computer system: executing the program instructions to select, for each cluster, the best-matched node pairs of the adjacent nodes using the Hausdorff distance to form the set of best-matched node pairs of the adjacent nodes for the set of clusters, the best-matched node pairs in the set having adjacent nodes from each of the two subgraphs; 23. The information management system of claim 22.
24. When assigning the adjacent nodes of the two central nodes of the two subgraphs to the groups by the node type, the groups include the adjacent nodes from both of the two subgraphs, and the computer system: placing the adjacent nodes from each subgraph of the two subgraphs into an initial group based on the node type of the adjacent nodes; To form the groups, select each initial group in the initial group that has adjacent nodes from both of the two subgraphs.
23. The information management system of claim 22, which executes said program instructions.
25. A computer program comprising: Identifying a first central node in the first subgraph and a second central node in the second subgraph; identifying a group of neighboring nodes having neighboring nodes from both the first subgraph and the second subgraph, the group of neighboring nodes within the group of neighboring nodes having the neighboring nodes of the same node type; identifying a best-matched node pair of adjacent nodes of each group of adjacent nodes to form a set of best-matched node pairs for a set of clusters, the best-matched node pair including a first adjacent node from the first subgraph and a second adjacent node from the second subgraph; determining whether the first core node and the second core node match using the first core node, the second core node, and the set of best-matching node pairs; A computer program for causing a method comprising:
Citation Information
Patent Citations
Document image database retrieval method, image feature vector extraction method, document image perusal system, medium which can be machine-read and image display method
JP1997237282A
Clustering device, clustering method, and clustering program
JP2015156163A
Method and apparatus for identifying similar sub-graphs in a network
US20100063973A1