Visual language framework for configurable identity resolution on large datasets

Abstract

A system for identity resolution for first-party data owners who wish to customize their identity graph creates prospect groups of records that could pertain to the same entity. Blocking keys are then created, using strict match parameters to minimize the prospect group size. In pair generation, duplicated entity comparisons are removed to reduce the compute required to perform the various operations. This is achieved by only generating pairs where the left value has a lower internal ID value than the right or vice versa. Next, the system generates a pair where both the left and right element are the same entity so that the system does not drop data through the resolution process.

Description

VISUAL LANGUAGE FRAMEWORK FOR CONFIGURABLE IDENTITY RESOLUTION ON LARGE DATASETSBACKGROUND

[0001] This application claims the benefit of US Provisional Patent Application No. 63 / 598,296, filed on November 13, 2023. Such application is incorporated by reference as if set forth herein in its entirety.

[0002] First-party data is defined as data that is collected by a data owner directly from entities. Data owners often have many distinct sources and "silos" (i.e., separate storage areas) of first-party data. It can be challenging to tie these first- party data sets to a consistent key in order to match the data together because they are often keyed to different identifiers (IDs) or source systems, and have incomplete or different data points to identify their accounts or records. In addition, the matching process required can be enormously expensive in terms of computational resources due to the large number of potential matches that are compared in order to achieve the resolution function. What is desired then is a simpler way for a data owner to configure and automate the process of creating a unified key representing an entity, and connecting to those source IDs for analytics or measurement or activation.

[0003] Prior resolution approaches have been either extremely rigid in producing specific entity resolution outcomes or heavily dependent on technical customization through coding. In both cases, extensive entity resolution expertise and experience is needed, which is often not available within a client's in-house personnel. The inventors hereof have recognized that a resolution framework and a user interface on top of the framework would abstract some of the technical problems specific toentity resolution that a client must address in order to build and apply a first-party identity graph customized to the client's entity data for the client's specific use case. Further, it would be desirable to develop a process for managing the unified key that significantly lowers the required number of match comparisons in orderto make the process computationally feasible within a time and resource budget applicable to real-world user applications.

[0004] References mentioned in this background section are not admitted to be prior art with respect to the present invention.SUMMARY

[0005] The present invention is directed to a system and method enabling a low / no- code identity resolution solution for first-party data owners who wish to customize an identity graph for their own data. The invention includes a resolution framework, a backend, and a user workflow editor interface. In certain implementations, the invention may feature drag-and-drop processes that have pre-defined operations for simplicity, as well as user-defined configurations that adjust the underlying code of processes. It may include filters that remove data elements based on existing or calculated attributes from source files, and export formats and destinations that deliver a first-party identity graph in a configurable location and format. This framework allows for the same level of customization, but without highly specialized entity resolution expertise in order to execute.

[0006] In a particular implementation of the present invention, the resolution process involves the conversion of a user-supplied configuration into a single instruction, such as but not limited to a SparkSQL statement in a Spark environment. Leveraging this technique within a Spark environment in this particularimplementation enables Spark's Catalyst optimizer to make the most efficient use of resources as it has visibility of the entire operation. In a blocking phase, the resolution process creates prospect groups of records that could pertain to the same entity. Blocking keys are then created, using strict match parameters to minimize the prospect group size and hence minimize the number of entity comparisons that need to be computed, thereby reducing compute resources required for this operation. The next step then is pair generation, which includes optimizations in certain implementations. Duplicated entity comparisons are removed to reduce the compute required to perform the various operations. This is achieved by only generating pairs where the left value has a lower internal ID value than the right. For example this avoids doing a match comparison on both pairs (entity_l, entity_2) and (entity_2, entity_l), since the comparison result is the same regardless of the ordering of the pair entities.

[0007] Next, the system generates a pair where both the left and right element are the same entity. This is done so that the system does not drop data through the resolution process, as it guarantees at least one match for each record. It is known that they are the same entity by definition, so the system may skip the matching phase (and hence the associated compute process) and pass the pair through to the next stage.

[0008] The two pair generation optimizations just described have reduced the number of entity comparisons in one real-world implementation from 59.2 million comparisons to 10.2 million comparisons, thus generating a reduction of 82% in required compute resources for the comparison function. This further translates to a significant reduction in the random access memory (RAM) space required to processthis data.

[0009] Additionally, based on the low / no-code user interface, the system has been able to create equivalent outcomes in building a client identity graph but reducing the necessary time frame from three to six months to two weeks. This includes making multiple iterations of graph rules and analysis.

[0010] These and other features, objects and advantages of the present invention will become better understood from a consideration of the following detailed description of the preferred embodiments and appended claims in conjunction with the drawings as described following:DRAWINGS

[0011] Fig. la is a workflow editor data flow diagram for an embodiment of the present invention.

[0012] Fig. lb is a data preparation data flow diagram for an embodiment of the present invention.

[0013] Fig. lc is a resolution data flow diagram for an embodiment of the present invention.

[0014] Fig. Id is an export data flow diagram for an embodiment of the present invention.

[0015] Fig. le is an iterative resolution reporting data flow diagram for an embodiment of the present invention.

[0016] Fig. 2 illustrates a user interface for a processor editor for data preparation according to an embodiment of the present invention.

[0017] Fig. 3 illustrates a user interface for a workflow editor according to an embodiment of the present invention.

[0018] Fig. 4 illustrates a user interface for an entity mapping editor according to an embodiment of the present invention.

[0019] Fig. 5. illustrates a user interface for a resolution processor configuration editor according to an embodiment of the present invention.

[0020] Fig. 6 illustrates a user interface for an export process configuration editor according to an embodiment of the present invention.

[0021] Fig. 7 is a data flow diagram showing an example match cascade according to an embodiment of the present invention.

[0022] Fig. 8 is a flow diagram showing key enrichment according to an embodiment of the present invention.

[0023] Fig. 9 illustrates a user interface for a data preparation dashboard according to an embodiment of the present invention.

[0024] Fig. 10 illustrates a user interface for a detailed view of imported data records according to an embodiment of the present invention.

[0025] Fig. 11 illustrates a user interface for a first-party data consolidation report according to an embodiment of the present invention.

[0026] Fig. 12 is an identity graph diagram illustrating interaction between multiple contributors in a first-party data graph according to an embodiment of the present invention.

[0027] Fig. 13 is a computer system for implementing a portion of the compute environment according to an embodiment of the present invention.DETAILED DESCRIPTION

[0029] Before the present invention is described in further detail, it should beunderstood that the invention is not limited to the particular embodiments described, and that the terms used in describing the particular embodiments are for the purpose of describing those particular embodiments only, and are not intended to be limiting, since the scope of the present invention will be limited only by the claims.

[0030] An overview of the entire process flow according to an embodiment of the invention as will be described herein is shown in Figs, la, with more detailed portions of the flow illustrated in Figs, lb-le. The process will be described with reference to these figures, and also with reference to the figures illustrating user interfaces to interact with these processes.

[0031] As a preliminary step, technical users may interact with the system through process editor user interface 50 of Fig. 2. The process editor allows technical users to leverage code to interact with the framework to create specialized tasks and functions. It supports various types of code across different infrastructure to facilitate a flexible system of customization. This access point provides sophisticated implementation of testing and optimization. This underlying system allows the reuse of established methods, significantly reducing the time and cost of deployment across use cases. The processes are code agnostic, which allows various methods of functions to be used together to create a cohesive output. This environment is thus a collaborative environment for engineers of different expertise (e.g., data engineering, machine learning, etc.).

[0032] The workflow editor interface 52, shown in Fig. 3, is an accessible system that provides a clear picture of the entire identity resolution process for the end user. It allows the construction of sophisticated identity resolution pipelines without theneed of highly technical users. This no-code system for identity resolution creates configurable identity resolution pipelines that can meet a variety of use cases. At the left of workflow editor interface 52, there is seen a process list panel. This shows available process templates that a user can use for various stages throughout an identity resolution workflow. These include process templates for data preparation 10, resolution 12, and export 14, as shown in Fig. la. In the illustrated example, the data preparation 10 processes are standard customer data source and special customer ingest. The resolution 12 processes in the illustrated example are standard individual resolution, probabilistic match model, custom householding model, and random forest modeling. The export 14 process options are GCP® export, Azure® export, and Snowflake® export. These are only examples, as it will be understood that many alternative processes may be made available through the system as described herein.

[0033] In order to configure a particular workflow through workflow editor interface 52, the non-technical user may simply drag processes from the process list panel at the left into the corresponding canvas section to the right. Fig. 3 provides an example of some processes being added to the canvas, although many other configurations are possible. Once a process has been added to the canvas, it must be configured to target the specific data within the desired workflow.

[0034] The workflow name (in this example, "Customer Household Resolution") is displayed above the canvas. Access to the metadata associated with data being used in a particular workflow is available through the "information" icon as shown in Fig. 3.

[0035] Enterprise data is often spread across multiple, disparate data sources 22that typically require specialized skills and bespoke tools to unify in order to create organizational value. Data preparation 10 processes, beginning at the configure data preparation user interface shown in Fig. 4, can be configured without the need to leverage code, while still supporting necessary data transformations to normalize incoming raw data. The flow for this process is shown in greater detail in Fig lb.

[0036] Users may read in their source data 24 from the various data sources. A particular identity type selection 26 may be made. Users map existing data to a schema of "entity" types 20 that provide built-in context without the need for advanced data engineering skills or identity resolution expertise. This mapping also creates signals within the framework that provide the user with relevant recommendations for effective resolution techniques specific to the user's data at lookup 30.

[0037] Mapping of data using identity type selection 26 is performed at mapping step 28, and a first verification and acceptance step 32 is performed. A variety of data transformations can be applied across fields to format raw data and achieve a consistent quality and structure of data to be utilized in a resolution process.

[0038] Third party hygiene services 34 can be integrated to validate various data in each data preparation process. This may include, for example, standardization of addresses to a particular format, use of consistent abbreviations, etc. A second verification and acceptance step 36 may then be performed to ensure the success of the previous step.

[0039] Data may optionally be "enriched" with additional relevant data fields for a particular data entity. For example, purchasing propensities of a particular consumer may be added as enrichment data to data pertaining to a group ofconsumers. An enrichment service may be selected at step 40 if it is determined that enrichment is required or desired at step 38, and then the data is enriched at step 42. A third verification and acceptance step 44 may then be performed to ensure that data consistency is maintained in order to complete data preparation process 10. The result of this enrichment process is that many keys are associated to entity combinations, which can then be configured in match rules and applied to eligible pairs as configured by the user, without adopting the reference identity graph cluster.

[0040] Referring now to Fig. 4, it may be seen how the non-technical user can control the various processes described with reference to Fig. lb. The user configures each data preparation 10 process to address the specific characteristics of the data in question. Data can be drawn from numerous sources regardless of location, format, and quality. At the left of workflow editor user interface 44, the stages of the process may be tracked. In the illustration, the user is currently at the entity mapping stage.

[0041] Users can map their data to known entities for identity resolution. Available data transformation accounts for any inconsistencies that exist in incoming data, including for example data normalization and hygiene 34. Data can be normalized through various custom hygiene services to provide optimal results in identity resolution. The entity mapping section of configure data preparation user interface 54 shows the process of configuration and transformation. Available data fields in this particular example are email, address, individual name, customer ID, and home telephone number.

[0042] Resolution process configuration may be performed through the resolutionprocess configuration 56 user interface of Fig. 5. Processes leverage "match rules to consolidate data to meet a variety of use cases. These rules can produce person and household identities with different levels of strictness.

[0043] The system uses the signals created in data preparation to recommend match rules based on the available data types present across the various sources. This reduces the need for the user to have a deep understanding of complex identity resolution techniques and facilitates fast iteration to achieve effective outcomes.

[0044] A user can edit or create match rules through a simple interface that abstracts the complexity of methods and provides simple choices that are easy to understand by various levels of technical skill.

[0045] Fig. lc provides a more detailed flow for the resolution 12 process that is configured through the resolution process configuration 56 user interface of Fig. 5. Data is received at merge and deduplicate step 60, which prepares the data for further processing. At blocking step 62, the resolution process creates prospect groups of records that could pertain to the same entity. Blocking keys are then created, using strict match parameters. These parameters minimize the prospect group size and thereby minimize the number of entity comparisons that need to be computed.

[0046] The next step then is pair generation 68, which includes optimizations in certain implementations. Duplicated entity comparisons are removed to reduce the compute required to perform the various operations at this step. This is achieved by only generating pairs where the left value has a lower internal ID value than the right. For example, this avoids doing a match comparison on both pairs (entity_l, entity_2) and (entity_2, entity_l), since the comparison result is the same regardlessof the ordering of the pair entities.

[0047] Next, the system generates a pair where both the left and right element are the same entity. This is done so that the system does not drop data through the resolution process, so that there is always at least one match for each record. It is known that they are the same entity by definition, so the system may skip the matching phase 66 (and hence the associated compute process) and pass the pair through to the next stage.

[0048] Match phase 66 is implemented using a predictive model that has been trained in the user interface, trained externally and exported as a .pmml format model, or a proprietary model, for example. The model sits in the same part of the resolution process, and is invoked in conjunction with or in place of rules after pair generation to determine if pairs should be clustered to a new entity. This can occur for person, household, account, or other entity resolution precision types. The model is called for each pair as a predict function, which will return a score. A rule will then merge pairs or keep pairs apart based on the threshold configured by the user for matching.

[0049] Connect components step 64 applies the connected components algorithm, such as is available in the Apache Spark® environment, to label each connected component in the identity graph with the ID of its lowest-numbered vertex. Entity IDs (according to the particular implementation) are assigned at entity ID assignment step 70. At step 72, the knowledge base is written to a warehouse, denoted in Fig. lc as Knowledge Base 78. Finally, statistics generated during the resolution run are written at step 74 to statistics per resolution run database 76.

[0050] Once the resolution process is complete, a user interface for export processconfiguration 58 is shown in Fig. 6. Export is the process of extracting data from a generation of the first-party identity graph for an external view or application downstream. At the application phase, a business user or engineer can dictate the fields, identifiers (IDs), structure, and connectivity of data that is extracted, using either a step-by-step wizard in the user interface (Ul) or by writing SQL queries and scheduling those queries to run based on a schedule or in response to a trigger.

[0051] A first-party graph might have many "export" functions that pull data in the format, cadence, and specifications of each downstream application or use case. For example, if entities can be connected to multiple IDs within the graph build, but are not allowed or supported for certain downstream use cases, the user can specify how tie-breaking or entity selection will occur at the export stage for each extract that will occur.

[0052] Fig. Id provides a data flow for the export process configured by the user interface for export process configuration 58 as shown in Fig. 6. The current resolution result is reviewed at step 90, and compared to the most recent resolution result at step 92. Both of these processes use information from the statistics per resolution run database 76, as written at step 74. Select mapping or matching to adjust step 94 is how the process for tie-breaking or entity selection may be implemented from user interface for export process configuration 58.

[0053] The system, as discussed above with reference to Fig. lc, can use a third- party reference identity graph to enrich records with keys from a third-party identity reference graph in order to increase the rate of consolidation, bring in household data, and enable matching entities that otherwise will not share a common join key or any overlapping data in the source data sets.

[0054] In other models of matching to a reference graph, the user sacrifices configurability and the value of their first-party data because they will match each record to the best record in the reference data set, and use that link as a key for ID assignment. Fig. 7 provides an illustration of a standard match cascade to determine the best key in the graph for an entire record.

[0055] In other models, a record attempts to match down a cascade to find the best key from the reference graph, and stops after one or more keys have been retrieved from the reference source. If there are no matches to be found, then no key is returned. The result of this is that the source dataset will be keyed with the reference source ID, and entities can be grouped or joined based on that ID. The limitation of this approach is that it will not account for tiebreakers when multiple keys could match to a record, or cases in which touchpoints may be shared, which could result in overmatching. The customer data has been fit into the reference graph definition of an entity cluster, and does not alter or change such clusters.

[0056] The purpose of the framework is to create the best cluster of entities for the customer given the configurations and business needs that the user has in mind when configuring a graph in the platform. Therefore, they do not lock into the reference source definition. In order to get the maximum number of keys back from the reference source, the framework will enrich multiple entity combinations with keys from the reference source during the data preparation phase.

[0057] Cluster process 62 of Fig. 8 provides an illustration of the process for creating the best cluster. Beginning at the first-party source data, various combinations of attributes from each entity are matched against the third-party identity graph. The first column of matches following the third-party identity graph to the rightrepresents a full match against the third-party identity graph. If there is no full match, then processing proceeds, where the next two columns show partial matches in this example. The right-most column is simply the return of the first-party data if no match is found in the third-party identity graph.

[0058] Rule sets or algorithms for identity resolution cannot handle every edge case, so there is also a process supported in the tool to override the configured split and merges of entities that would have been resolved according to the configured graph rules for cases such as customer service requests to pull apart conflated records, or to bring accounts together. Overriding these use cases one-off allows identity graph owners to quickly resolve edge case problems without making rule changes that would have significant collateral impacts on other cases.

[0059] In the tool, overrides are added to a "forced merge" or "forced split" table by the record IDs, which will override the rules process for each graph build. Figs. 9 and 10 provide data preparation record trends interface 62 and import details interface 64, respectively, for the data through the data preparation dashboard. In this way, the user may see, for example, the number of rejected records and the number of records rejected related to a particular acceptance rule. Fig. 11 provides feedback on resolution results, showing, for example, the total number of records, the unique attribute count across these records, and the portrait fill rate distribution.

[0060] The tool can also be configured to apply the opt out and deletion requirements that companies must apply to their data across various jurisdictions. Certain sources are marked as "consent" sources, and then opt out behavior can be specified to apply "hard" or "soft" deletes. A hard delete is a full removal of all current and historical data associated with an individual, whereas a "soft" deletemaintains that record and adds it to a filter group, to ensure it is not returned in the graph or added again unless the consent status changes.

[0061] In most collaboration cases, customers directly match their personally identifiable information ( PI I ) between parties in a clean room, or do hashed email matching between multiple parties, but they do not perform person-based collaboration without a third-party reference identity graph.

[0062] The system can be used to create a stable consensus identifier between multiple parties to enable person-based collaboration on each party's primary identifier. In this model, each data controllerthat participates in collaboration contributes pseudonymous data touch points into a cooperative graph of graphs in order to facilitate person- or household-based collaboration and identity resolution between parties.

[0063] An illustrative example of this process is shown in collaborative match framework example 70 of Fig. 12. The process begins with four sources, related to potential ID clusters A-D. At the end of the process, these are resolved down to two entities, Pl and P2, in the graph. It is seen that entities A and B were in fact the same entity, now matched together at entity Pl. Additionally, it will be seen in this example that an overconnected email address that is suppressed. New types of IDs may be created with links to other IDs.

[0064] The end result of the process shown in Fig. 12 is that stable entity (e.g., person and household) identifiers are created that allow stable measurement as matching data is received across multiple sources. When collaborating, each party traverses the identity graph to join its data with any partner in the model via the clean room, and then receives data exported or reported grouped to its first-partyidentifier.

[0065] Embodiments of the invention may be implemented in a Spark environment, although the invention is not limited to this environment. Based on comparisons to running the same customer's data through the same identity resolution processes, the embodiment of the invention described herein and implemented in a Spark environment has resulted in an approximately 800% performance increase in build time and computational cost for the processing of data to build a customer first- party identity graph compared to the previously existing process used for a similar outcome without extensive configuration. This means that the system can run the same customer workflow in this new architecture on less than half of the provisioned nodes, at less than one-third of the execution time, compared to the legacy architecture.

[0066] In a previous model, custom objects and Spark were leveraged in a framework to conduct identity resolution. These objects were individually updated and configured by users, and inherited arguments from one another. This implementation technique resulted in a large number of objects that the Spark framework could not natively understand and hence the Spark framework could not internally optimize the execution of each custom object.

[0067] The new model uses a similar build process but translates the user's configuration into a complete execution flow leveraging Spark's native DataFrame objects. This enables Spark to optimize the workflow into the most efficient execution plan. The model further optimizes execution by limiting the number of pairs generated before the connected components algorithm. This is achieved by deduplicating pairs as well as skipping computationally expensive steps (particularlyfuzzy match) when not required.

[0068] Additionally, based on the low / no-code user interface, the embodiment of the invention described herein demonstrates equivalent customer outcomes in building a customer identity graph from three to six months to two weeks for users to get equivalent results. This includes making multiple iterations of graph rules and analysis.

[0069] Another advantage of the embodiment described herein is that customers may store and process data in their own compute cloud while still leveraging the product features. The primary customer value is that they do not have to share their data with the solution provider but can still leverage the solution provider's product.

[0070] The methods described herein may in various embodiments be implemented by any combination of hardware and software. For example, in one embodiment, the methods may be implemented by a computer system or a collection of computer systems, each of which includes one or more hardware processors executing program instructions stored on a computer-readable physical storage medium coupled to the hardware processors. The program instructions may implement the functionality described herein (e.g., the functionality of various hardware servers and other components that implement the network-based cloud and non-cloud computing resources described herein). The various methods as illustrated in the figures and described herein represent example implementations. The order of any method may be changed, and various elements may be added, modified, or omitted.

[0071] Fig. 13 is a block diagram illustrating an example computer hardware system, according to various embodiments. Computer system 140 may implement a hardware portion of a cloud computing system or non-cloud computing system, asforming parts of the various implementations of the present invention. Computer system 140 may be any of various types of hardware devices, including, but not limited to, a commodity server, personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, handheld computer, workstation, network computer, a consumer device, application server, physical storage device, telephone, mobile telephone, or in general any type of computing node, compute node, compute device, and / or hardware computing device.

[0072] Computer system 140 includes one or more hardware processors 140a, 141b...141n (any of which may include multiple processing cores, which may be single or multi-threaded) coupled to a physical system memory 142 via an input / output (I / O) interface 144. Computer system 140 further may include a network interface 146 coupled to I / O interface 144. In various embodiments, computer system 140 may be a single processor system including one hardware processor 141a, or a multiprocessor system including multiple hardware processors 141a, 141b...l41n. Processors 141a, etc. may be any suitable processors capable of executing computing instructions. For example, in various embodiments, processors 141a, etc. may be general-purpose or embedded processors implementing any of a variety of instruction set architectures. In multiprocessor systems, each of processors 141a, etc. may commonly, but not necessarily, implement the same instruction set. The computer system 140 also includes one or more hardware network communication devices (e.g., network interface 146) for communicating with other systems and / or components over a communications network, such as a local area network, wide area network, or the Internet. For example, a client application executing on system 140 may use network interface 146 to communicatewith a server application executing on a single hardware server or on a cluster of hardware servers that implement one or more of the components of the systems described herein in a cloud computing or non-cloud computing environment as implemented in various sub-systems. In another example, an instance of a server application executing on computer system 140 may use network interface 146 to communicate with other instances of an application that may be implemented on other computer systems.

[0073] In the illustrated embodiment, computer system 140 also includes one or more physical persistent storage devices 148 and / or one or more I / O devices 150. In various embodiments, persistent storage devices 148 may correspond to disk drives, tape drives, solid-state memory or drives, other mass storage devices, or any other persistent storage devices. Computer system 140 (or a distributed application or operating system operating thereon) may store instructions and / or data in persistent storage devices 148, as desired, and may retrieve the stored instructions and / or data as needed. For example, in some embodiments, computer system 140 may implement one or more nodes of a control plane or control system, and persistent storage 148 may include the solid-state drives (SSDs) attached to that server node. Multiple computer systems 140 may share the same persistent storage devices 148 or may share a pool of persistent storage devices, with the devices in the pool representing the same or different storage technologies, including such technologies as described above.

[0074] Computer system 140 includes one or more physical system memories 142 that may store code / instructions 143 and data 145 accessible by processor(s) 141a, etc. The system memories 142 may include multiple levels of memory and memorycaches in a system designed to swap information in memories based on access speed, for example. The interleaving and swapping may extend to persistent storage devices 148 in a virtual memory implementation, where memory space is mapped onto the persistent storage devices 148. The technologies used to implement the system memories 142 may include, by way of example, static random-access memory (RAM), dynamic RAM, read-only memory (ROM), non-volatile memory, solid-state memory, or flash-type memory. As with persistent storage devices 148, multiple computer systems 140 may share the same system memories 142 or may share a pool of system memories 142. System memory or memories 142 may contain program instructions 143 that are executable by processor(s) 141a, etc. to implement the routines described herein.

[0075] In various embodiments, program instructions 143 may be encoded in binary, Assembly language, any interpreted language such as Java, compiled languages such as C / C++, or in any combination thereof; the particular languages given here are only examples. In some embodiments, program instructions 143 may implement multiple separate clients, server nodes, and / or other components.

[0076] In some implementations, program instructions 143 may include instructions executable to implement an operating system (not shown), which may be any of various operating systems, such as UNIX, LINUX, Solaris™, MacOS™, or Microsoft Windows™. Any or all of program instructions 143 may be provided as a computer program product, or software, that may include a non-transitory computer-readable storage medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to various implementations. A non-transitory computer-readable storage medium mayinclude any mechanism for storing information in a form (e.g., software or processing application) readable by a machine (e.g., a physical computer). Generally speaking, a non-transitory computer-accessible medium may include computer- readable storage media or memory media such as magnetic or optical media, e.g., disk or DVD / CD-ROM, coupled to or in communication with computer system 140 via I / O interface 144. A non-transitory computer-readable storage medium may also include any volatile or non-volatile media such as RAM or ROM that may be included in some embodiments of computer system 140 as system memory 142 or another type of memory. In other implementations, program instructions may be communicated using optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) conveyed via a communication medium such as a network and / or a wired or wireless link, such as may be implemented via network interface 606. Network interface 146 may be used to interface with other devices 142, which may include other computer systems or any type of external electronic device.

[0077] In some embodiments, system memory 142 may include data store 145, as described herein. In general, system memory 142 and persistent storage 148 may be accessible on other devices 142 through a network and may store data blocks, replicas of data blocks, metadata associated with data blocks, and / or their state, database configuration information, and / or any other information usable in implementing the routines described herein.

[0078] In one embodiment, I / O interface 144 may coordinate I / O traffic between processors 141a, etc., system memory 142, and any peripheral devices in the system, including through network interface 146 or other peripheral interfaces. In someembodiments, I / O interface 144 may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 142) into a format suitable for use by another component (e.g., processors 141a, etc.). In some embodiments, I / O interface 144 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, as examples. Also, in some embodiments, some or all of the functionality of I / O interface 144, such as an interface to system memory 142, may be incorporated directly into processor(s) 141a, etc.

[0079] Network interface 146 may allow data to be exchanged between computer system 140 and other devices attached to a network, such as other computer systems (which may implement one or more storage system server nodes, primary nodes, read-only node nodes, and / or clients of the database systems described herein), for example. In addition, I / O interface 144 may allow communication between computer system 140 and various I / O devices 150 and / or remote storage 148. Input / output devices 150 may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or retrieving data by one or more computer systems 140. These may connect directly to a particular computer system 140 or generally connect to multiple computer systems 140 in a cloud computing environment, grid computing environment, or other system involving multiple computer systems 140. Multiple input / output devices 150 may be present in communication with computer system 140 or may be distributed on various nodes of a distributed system that includes computer system 140. In someembodiments, similar input / output devices may be separate from computer system 140 and may interact with one or more nodes of a distributed system that includes computer system 140 through a wired or wireless connection, such as over network interface 146. Network interface 146 may commonly support one or more wireless networking protocols (e.g., Wi-Fi / IEEE 802.11, or another wireless networking standard). Network interface 146 may support communication via any suitable wired or wireless general data networks, such as other types of Ethernet networks, for example. Additionally, network interface 146 may support communication via telecommunications / telephony networks such as analog voice networks or digital fiber communications networks, via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and / or protocol. In various embodiments, computer system 140 may include more, fewer, or different components (e.g., displays, video cards, audio cards, peripheral devices, or an Ethernet interface).

[0080] Any of the distributed system embodiments described herein, or any of their components, may be implemented as one or more network-based services in the cloud computing environment. For example, a read-write node and / or read-only nodes within the database tier of a hardware database system may present database services and / or other types of physical data storage services that employ the distributed storage systems described herein to clients as network-based services. In some embodiments, a network-based service may be implemented by a software and / or hardware system designed to support interoperable machine-to-machine interaction over a network. A web service may have an interface described in a machine-processable format. Other systems may interact with the network-basedservice in a manner prescribed by the description of the network-based service's interface. For example, the network-based service may define various operations that other systems may invoke, and may define a particular application programming interface (API) to which other systems may be expected to conform when requesting the various operations.

[0081] In various embodiments, a network-based service may be requested or invoked through the use of a message that includes parameters and / or data associated with the network-based services request. Such a message may be formatted according to a particular markup language such as Extensible Markup Language (XML), and / or may be encapsulated using a protocol. To perform a network-based services request, a network-based services client may assemble a message including the request and convey the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the web service, using an Internet-based application layer transfer protocol such as Hypertext Transfer Protocol (HTTP).

[0082] In some embodiments, network-based services may be implemented using Representational State Transfer (REST) techniques rather than message-based techniques. For example, a network-based service implemented according to a REST technique may be invoked through parameters included within an HTTP method such as PUT, GET, or DELETE.

[0083] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing ofthe present invention, a limited number of the exemplary methods and materials are described herein. It will be apparent to those skilled in the art that many more modifications are possible without departing from the inventive concepts herein.

[0084] All terms used herein should be interpreted in the broadest possible manner consistent with the context. In particular, the terms "comprises" and "comprising" should be interpreted as referring to elements, components, or steps in a nonexclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.

[0085] When a grouping is used herein, all individual members of the group and all combinations and subcombinations possible of the group are intended to be individually included.

[0086] When a range is stated herein, the range is intended to include all sub-ranges within the range, as well as all individual points within the range.

[0087] When "about,” "approximately," or like terms are used herein, they are intended to include amounts, measurements, or the like that do not depart significantly from the expressly stated amount, measurement, or the like, such that the stated purpose of the apparatus or process is not lost.

[0088] All references cited herein are hereby incorporated by reference to the extent that there is no inconsistency with the disclosure of this specification.

[0089] The present invention has been described with reference to certain preferred and alternative embodiments that are intended to be exemplary only and not limiting to the full scope of the present invention, as set forth in the appended claims.

Claims

We claim:

1. A computerized identity resolution method, comprising the steps of: generating a workflow editor interface configured to facilitate construction of an identity resolution pipeline in a no-code environment, wherein the workflow editor interface comprises a workflow editor process list panel comprising a plurality of process templates and a workflow editor canvas configured to receive one or more of the plurality of workflow editor process templates to form the identity resolution pipeline; generating a configure data preparation user interface configured to receive at least one entity mapping based on at least one identity type selection and at least one scheme of entity types applied against a data set; generating a resolution process configuration user interface configured to display recommended match rules for the data set based on available data types present within the data set; generating a first-party identity data graph and perform a matching process wherein various combinations of attributes for each entity of a plurality of entities from the first-party identity data graph are matched against a third-party data graph to generate a full match if a match is found in the third-party identity data graph, or if no full match is generated to generate a partial match if a partial match is found in the third-party identity data graph, and return the first-party data only if no full match and no partial match is found in the third-party identity data graph; and generating an export process configuration user interface configured to facilitate the extraction of data from the first-party identity data graph.

2. The computerized method of claim 1, wherein the workflow user interface further comprises an information icon configured to provide metadata associated with the data set.

3. The computerized method of claim 2, wherein the configure data preparation user interface is further configured to enrich the data set with additional relevant data fields.

4. The computerized method of claim 3, further comprising performing a blocking step to create prospects groups of records from within the data set that could pertain to a same entity.

5. The computerized method of claim 4, further comprising creating blocking keys to minimize the size of the prospect groups of records to minimize a required number of entity comparisons.

6. The computerized method of claim 5, further comprising performing pair generation within the data set to generate a set of matched pairs.

7. The computerized method of claim 6, further comprising removing duplicates within the set of matched pairs by only generating pairs wherein a first value within each pair of the set of matched pairs has a lower internal ID value than a second value within each pair of the set of matched pairs.

8. The computerized method of claim 7 , further comprising generating pairs wherein both the first value and the second value are the same entity, whereby the computerized method does not drop any data through the identity resolution method.

9. The computerized method of claim 8, further comprising performing a match phase based on a matching model to determine if pairs should be clusteredwith a new entity based on a returned score, and cluster with the new entity if the returned score surpasses a threshold score.

10. The computerized method of claim 9, wherein the resolution process configuration user interface further facilitates application of an override to provide a forced merge or forced split within the first-party identity data graph.

11. A machine comprising one or more computer processors and a memory space having instructions stored therein, the instructions, when executed by the one or more computer processors, causing the one or more computer processors to: generate a workflow editor interface configured to facilitate the construction of an identity resolution pipeline in a no-code environment, wherein the workflow editor interface comprises a workflow editor process list panel comprising a plurality of process templates and a workflow editor canvas configured to receive one or more of the plurality of workflow editor process templates to form the identity resolution pipeline; generate a configure data preparation user interface configured to receive at least one entity mapping based on at least one identity type selection and at least one scheme of entity types applied against a data set; generate a resolution process configuration user interface configured to display recommended match rules for the data set based on available data types present within the data set; generate a first-party identity data graph and perform a matching process wherein various combinations of attributes for each entity from the first-party identity data graph are matched against a third-party data graph to generate a fullmatch if a match is found in the third-party identity data graph, or if no full match is generated to generate a partial match if a partial match is found in the third-party identity data graph, and return the first-party data only if no full match and no partial match is found in the third-party identity data graph; and generate an export process configuration user interface configured to facilitate the extraction of data from the first-party identity data graph.

12. The machine of claim 11, wherein the workflow user interface further comprises an information icon configured to provide metadata associated with the data set.

13. The machine of claim 12, wherein the configure data preparation user interface is further configured to enrich the data set with additional relevant data fields.

14. The machine of claim 13, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to perform a blocking step to create prospects groups of records from within the data set that could pertain to the same entity.

15. The machine of claim 14, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to create blocking keys to minimize the size of the prospect groups of records to minimize a required number of entity comparisons.

16. The machine of claim 15, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to perform pair generation within the data set to generate a set of matched pairs.

17. The machine of claim 16, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to remove duplicates within the set of matched pairs by only generating pairs wherein a first value within each pair of the set of matched pairs has a lower internal ID value than a second value within each pair of the set of matched pairs.

18. The machine of claim 17, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to generate pairs wherein both the first value and the second value are the same entity, whereby the computerized method does not drop any data through the identity resolution method.

19. The machine of claim 18, further wherein the instructions, when executed by the one or more computer processors, further cause the one or more computer processors to perform a match phase based on a matching model to determine if pairs should be clustered with a new entity based on a returned score, and cluster with the new entity of the returned score surpasses a threshold score.

20. The machine of claim 19, further wherein the resolution process configuration user interface facilitates the application of an override to provide a forced merge or forced split within the first-party identity graph.

Images