Flexible Entity Resolution Network
By combining a large language model and a downstream neural network classifier, matching data records in the computing system are identified and merged, solving the problem of inefficiency in processing heterogeneous data and achieving more efficient resource utilization and accurate data merging.
Patent Information
- Application Number
- CN202580014947.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-03-25
- Filing Date
- 2025-02-18
- Publication Date
- 2026-11-17
AI Technical Summary
The computing system is inefficient when processing large amounts of heterogeneous data, wastes computing resources, and has difficulty effectively identifying and merging multiple data records of the same entity.
By combining a large language model and a downstream neural network classifier, matching data records within a data record set are identified and merged using parallel-executed matching rules and machine learning models. This real-time matching and merging mechanism optimizes the efficiency and accuracy of the computing system.
It improves the efficiency and accuracy of computing systems when processing heterogeneous data, reduces the demand for memory, processors and network bandwidth, and achieves higher computing speed and lower resource consumption.
Smart Images

Figure CN123125602A_ABST
Abstract
Description
Background Technology
[0001] Computing systems routinely store and process large amounts of heterogeneous data. Processing such large volumes of data consumes significant computing resources (e.g., memory, processing speed, and network bandwidth). Computing systems are also often highly inefficient and wasteful of computing resources when processing such large volumes of data. For example, the data may include different data records for the same entity (e.g., multiple data records for the same employee in an organization), causing the computing system to process far more data records than needed. This can lead to increased computing requirements (e.g., memory, storage devices, processors, network bandwidth) and reduced computing speed and efficiency, as well as other technical problems. Attached Figure Description
[0002] Figure 1 The connected data platform is described.
[0003] Figure 2 The environment for integrating the central nervous system is described.
[0004] Figure 3 A three-layer model is depicted in some embodiments.
[0005] Figure 4 A diagram depicting some examples of entity types, relationship types, and event metadata.
[0006] Figure 5 A flowchart facilitating dynamic matching is depicted.
[0007] Figure 6 Depicting Figure 5 A graphical representation of the Data Change Request (DCR) workflow review process.
[0008] Figure 7 and Figure 8 This is an example of a data change request review panel for the user interface (UI).
[0009] Figure 9 The Additional Details tab in the Data Change Request Review panel is described.
[0010] Figure 10 The interface for creating new characters is described.
[0011] Figure 11 The interface for editing users is described.
[0012] Figure 12 The interface for requesting data changes is described.
[0013] Figure 13 The interface for data change requests, including the "Cancel Reject" option, is described.
[0014] Figure 14 The interface for reviewing data change requests that describe the state of a relationship is described.
[0015] Figure 15 The interface for requesting data changes is described.
[0016] Figure 16 , Figure 17 and Figure 18 It describes changes to relationships and their attributes, or the creation or deletion of relationships.
[0017] Figure 19 A diagram of an example flexible entity resolution network system is depicted.
[0018] Figure 20 A schematic diagram of an exemplary AI-based matching system is depicted for performing machine learning-based matching analysis and prediction using generative AI models (e.g., large language models) and downstream neural network classifiers.
[0019] Figure 21 A flowchart depicts an example method for flexible entity parsing based on AI.
[0020] Figure 22 A flowchart is shown for an example method of interactive parallelized multimodal matching.
[0021] Figure 23 A flowchart is shown illustrating an example method for analyzing matching rules and generating machine learning-based matching rule recommendations.
[0022] Figure 24 A flowchart is depicted illustrating an example method for AI-based flexible entity parsing using one or more approximator networks. Detailed Implementation
[0023] The claimed solution, rooted in computer technology, overcomes problems particularly prevalent in the field of computer technology. In various embodiments, the computing system is configured to provide a multi-class concatenation of a large language model and an N-order neural network, which identifies matching data records within a set of data records and then merges the matching records. More specifically, the computing system can use a combination of a domain-agnostic large language model and a downstream neural network classifier to identify matching data records, which is not possible using other machine learning or rule-based entity resolution systems. In one example, the computing system receives an entity resolution request. The entity resolution request may indicate a first entity (e.g., “Александр Иванов” with the address “135 West Union Avenue, Baltimore, 21201, USA”) and a second entity (e.g., “Dr. Sarah Jackson” with the address “123 West Union Avenue, Baltimore, 99801, USA”). For example, a data administrator may provide an entity resolution request to help determine whether the entities are the same or different.
[0024] Using entity resolution requests, the computing system can identify attributes in the data model. For example, attributes may include the name attribute (e.g., prefix, first name, middle name, last name, suffix) of the name object in the individual entity model. The computing system can then identify other attributes in the data model and / or other data models. For example, other attributes may include the address attribute (e.g., address line 1, address line 2, city, etc.) of the address object in the individual entity model.
[0025] The computational system can then create strings based on the attributes of the identified data model. For example, the computational system can concatenate a first set of attributes to create a first string and concatenate a second set of attributes to create a second string. The computational system can then generate domain-agnostic large language model (LAM) hints based on the first string and other LLM hints based on the second string. The computational system can provide LLM hints to a domain-agnostic large language model (e.g., a Flan-T5 XL model that has not yet been trained on any domain-specific dataset or customer dataset, etc.). The LLM can generate LLM results. For example, the results can include natural language results such as "Yes, it matches..." or "No, it does not match." In some embodiments, the results can include feature scores (e.g., full name: 0.75, address lines 1 & 2: 0.63, address city: 0.94, and name list: 0.47).
[0026] In some embodiments, the computational system may generate and / or execute an approximator network (e.g., an approximator network model). The approximator network may use a large language model (e.g., a domain-agnostic large language model) as a labeling tool for the prompt. Using the original string used here to create the prompt, along with the large language model as a labeling tool, the computational system may create a smaller neural network whose purpose is to replicate the behavior of the large language model only on that single problem (e.g., without any ability to do anything else).
[0027] In some embodiments, the approximator network is used to simulate large language model cues, and the approximator network can be a fully connected neural network trained using stochastic gradient descent, standard techniques such as dropout, momentum, and / or early stopping. In this way, a computational system can replace a large language model (e.g., a domain-agnostic large language model) with an approximator network to improve speed and efficiency.
[0028] The computational system can also prune the results of domain-agnostic large language models. The system can prune the results to "yes" or "no." For example, the system can identify yes or no and remove the remainder of the result. This can, for example, prevent any type of illusion.
[0029] The computational system can also generate the final result using one or more downstream neural network classifiers based on the results of a pruned domain-agnostic large language model. For example, the final output could be a score based on the results of a first large language model and a second large language model. In some embodiments, the downstream neural network classifier can combine feature scores from the domain-agnostic large language model results to generate the final result.
[0030] The calculation system can then merge the first and second entities based on the final result. The calculation system can merge the first and second entities because they are considered the same entity based on the final result. For example, a data administrator can review the final result and determine that they are the same entity. In other examples, the calculation system can automatically determine that they are the same entity (e.g., based on a comparison with a threshold score).
[0031] In various embodiments, the computing system is configured to identify and merge matching data records within a set of data records. More specifically, even when the data records include different data structures, data formats, and / or information, the computing system can use both parallel-executed matching rules and machine learning models (e.g., large language models, downstream neural network classifiers, etc.) to identify different data records as potential matches. For example, the matching rules and machine learning models can be executed independently to independently determine potential matches. The system can present potential matching data records and indicate whether the match was determined based on the matching rules and / or the machine learning matching model. The user can then choose whether to merge the records based on this determination (e.g., because they believe the data records are a match). By providing both rule-based matching and machine learning-based matching in parallel, the system can identify matching data records more efficiently and accurately, reduce computational requirements (e.g., memory, storage devices, processors) for subsequent operations on the data records, and provide the user with (e.g., higher match confidence compared to systems based solely on rules or machine learning).
[0032] In some embodiments, the computing system can also be configured to analyze matching rules and their performance in a live production environment. For example, the computing system can identify redundant matching rules (e.g., matching rules that produce substantially similar results), matching rules with too broad a scope (e.g., matching rules that produce too many matching results), and matching rules with too narrow a scope (e.g., matching rules that produce too few matching results or matching rules that are never triggered). The computing system can then generate one or more recommendations based on one or more machine learning models to improve the performance of the matching rules and the matching results. For example, the computing system can recommend merging, deleting, adding, and modifying matching rules.
[0033] In various embodiments, the unique architecture enables efficient modeling of the entities, relationships, and interactions that typically form the basis of a business. These models provide insights, scalability, and manageability previously unavailable in existing technologies. It should be understood that, using the information models discussed herein, there is no need to consider any low-level physicalities of tables, foreign keys, or how data is stored.
[0034] Information models can be used as part of a multi-tenant platform. In a particular implementation, configuration resides in a layer above the platform (such as the RELTIO® platform) and natively enjoys capabilities provided by the platform, such as matching, merging, sanitizing, standardizing, and workflows. Entities established within tenants can be associated with custom and / or standard interactions of the platform. The ability to maintain and link these three types of data (i.e., entities, relationships, and interactions) within the platform and leverage their convergence in one place provides businesses with the power to model and understand.
[0035] In various embodiments, metadata configuration is based on an n-tier model. One example is a 3-tier model (e.g., the default layout). In some embodiments, each tier is represented by a JSON file (however, it should be understood that many different file structures, such as BSON or YAML, can be utilized).
[0036] Information models can be used as part of the connected multi-tenant systems. Figure 1 Platform 102 is described. Platform 102 enables seamless scaling across a wide range of operational or analytical use cases. Platform 102 can serve as the foundation for Master Data Management (MDM). A variety of integration options, including low-code / no-code solutions, allow for rapid deployment and time-to-value realization.
[0037] Figure 1 These are examples of the functionality of Platform 102 in some embodiments. Platform 102 can support best-in-class MDM capabilities, including identity resolution, data quality, dynamic survivability for context profiles, universal IDs across operations and tiers, knowledge graphs for managing relationships, progressive stitching for creating richer profiles, and governance capabilities. Furthermore, Platform 102 can support high-volume transactions, high-volume API calls, complex analytics, and backend jobs for automatically scaling any workload in a cloud environment. Accordingly, Platform 102 can leverage built-in NoSQL databases, Elasticsearch, Spark, and another AWS and GCP service across multiple regions to support high redundancy, fault tolerance, and availability.
[0038] In various embodiments, platform 102 is multi-domain and enables seamless integration of many types of data from many sources to create master profiles for any data entity (individual, organization, product, location). Users can create master profiles for consumers, B2B clients, products, assets, and sites, and connect them to view the full picture.
[0039] Platform 102 enables an API-first approach for data integration and orchestration. Users (e.g., tenants) can use APIs and various application-specific connectors to simplify integration. Additionally, in some embodiments, users can stream data to analytics or data science platforms for instant insights.
[0040] Figure 2The environment for the integration hub 202 is described. The integration hub 202 can connect to various data sources and downstream consumers. In some embodiments, the integration hub 202 has over 1000 connectors to properly build data pipelines. The integration hub 202 may include an intuitive drag-and-drop graphical interface to create simple, replicable pipelines for complex data extraction and transformation tasks. Utilizing pre-built community recipes for common use cases, users can establish integration workflows in just a few clicks.
[0041] Along with its built-in data loader, event streaming capabilities, data API, and partner connectors, Integration Hub 202 enables rapid linking to user systems using Platform 102. Integration Hub 202 allows users to build automated workflows within hours or days to exchange data with any number of SaaS applications relative to Platform 102. Faster integration enables quicker access to unified, trusted data to drive real-time business operations.
[0042] Figure 3 A three-tier model is depicted in some embodiments. In this three-tier model, only tier 3 (e.g., the top tier in an n-tier model) 302 (referred to as "L3") is accessible to clients. This tier is part of a tenant. Information associated with tier L3 302 can be retrieved from the tenant, edited, and applied back to the tenant using a configuration API.
[0043] Layer 302 typically inherits from Layer 204 (industry-centric layer), which in turn inherits from Layer 106 (industry-agnostic layer). Generally, Layer 302 references the Layer 204 container and inherits all data items (or "objects") from it. However, Layer 302 is not required to reference the Layer 204 container; it can exist independently.
[0044] L2 layer 304 can inherit objects from L1 layer 306. Although there is only a single set of L1 layer 306 objects, objects at L2 layer 304 can be grouped into industry-specific containers. Similar to L1 layer 306, the containers at L2 layer 304 can be controlled by product management and may not be accessible to customers.
[0045] Life Sciences is a good example of an L2-level 304 container. An L2-level 304 container can inherit the Organization Entity type from L1-level 306 (discussed further in this paper) and extend it to the Health Care Organization (HCO) type required in Life Sciences. Therefore, the HCO type enjoys all the attributes and other characteristics of the Organization type, but defines the additional attributes and characteristics required by the HCO.
[0046] L1 layer 306 may contain entities such as participants (abstract types) and locations. In some embodiments, L1 layer 306 contains a basic relationship type called HasAddress that links participant types to location types. L1 layer 306 also extends participant types to organizations and individuals (both non-abstract types).
[0047] There can be only one L1 layer 306, and its role is to define industry-agnostic objects that can be inherited and utilized by industry-specific layers located at L2 layer 304. This allows for the enhancement of objects in L1 layer 306, potentially impacting all clients. For example, if an additional attribute is added to the HasAddress relationship type, it will typically be immediately available to any client of the platform.
[0048] Any object can be defined in any layer. Often referred to as tenant configuration or metadata configuration, it is a merged configuration derived from inheritance across the three layers. In a particular implementation, metadata configuration merges simple, nested, and reference properties from all relevant layers. Values described in higher layers provide overriding control over values from lower layers. The number of layers does not affect inheritance.
[0049] Figure 4 A diagram illustrating some examples of entity types, relationship types, and event metadata is provided. Platform 102 enables object types to be entities, relationships, and interactions. Entity type 402 can be a category of entities. For example, "individual" is entity type 402, and "Alyssa" represents a specific instance of that entity type. Other common examples of entity types include "organization," "location," and "product."
[0050] Typically, entity types can be materialized in a single instance (such as the "Alyssa" example above). In other examples, L1 layer 306 could define an abstract "Participant" entity type with a small set of attributes. L1 layer 306 could then be configured to define "Individual" and "Organization" entity types, both inheriting from "Participant," being non-abstract, and adding additional attributes specific to their type and business functions. Continuing the concept of inheritance, in the L2 layer 304 Life Sciences container, an HCP entity (e.g., to represent a doctor) could be defined, inheriting from the "Individual" type but also defining a small set of attributes specific to the HCP concept. Thus, there are entity classifications "Participant," "Individual," or "HCP," and the resulting HCP entity type provides developers and users with aggregated attributes of "Participant," "Individual," and "HCP."
[0051] Once entity types are defined, users can link entities together in the data model using relationship types. Once entity types are defined, they can be linked by defining relationships between them. For example, a user can independently publish a relationship to link two entities together, or a user can mention the relationship in JSON, and then the JSON publishes the relationship and both entities at once.
[0052] Relationship type 404 describes a link or connection between two specific entities (e.g., entities 406 and 408). Relationship type 404, together with the described entities 406 and 408, forms a graph. Some common relationship types are organization to organization, affiliate of, partner of, individual to individual, parent / child of, report to, individual to organization / organization to individual, related to, employee / contractor of.
[0053] Platform 102 allows users to define metadata attributes and properties specific to relation types. Users can define any number of metadata attributes. Users can also define several properties specific to a relation type, such as name, description, direction (undirected, directed, bidirectional), start and end entities, etc. Properties of one relation type can be inherited from another relation type.
[0054] Hierarchies can be defined through the definition of relation subtypes. For example, if a user defines "Family" as a relation type, then the user can define "Parents" as a subtype. A hierarchy contains one or more relation types; all entities connected by these relations form the hierarchy. Entity A > HasChild(Entity B) > HasChild(Entity C). Then A, B, and C form the hierarchy. Within the same hierarchy, a user can add dependents as relations, and if entity D is a dependent of entity C, then A, B, C, and D all become part of a single hierarchy.
[0055] Interaction 410 is a lightweight object representing any kind of interaction or transaction. In a broader sense, interaction 410 represents an event that occurs at a specific moment (such as a retail purchase or measurement). It can also represent facts over a period of time, such as sales figures for June.
[0056] Interaction 410 can have multiple participants (entities) and can have different record lengths, columns, and formats. Attribute types can be used to define the data model. Therefore, users can build logical data models instead of relying on physical tables and foreign keys; define entities, relationships, and interactions with granular detail; make detailed data available to content and interaction designers; and provide business users with rich yet fluid search and navigation experiences.
[0057] In various embodiments, the four representations of attribute types include simple, nested, referenced, and analytic. A simple attribute type represents a single characteristic of an entity, relationship, or interaction. Nested, referenced, and analytic attribute types represent combinations or collections of simple sub-attribute types.
[0058] Nested property types are used to create collections of simple properties. For example, a phone number is a nested property. Sub-properties of a phone number typically include the number, type, area code, and extension. In the phone number example, the sub-properties are only meaningful when held together as a collection. When published as a nested property, the entire collection represents a single instance or value of the nested property. Publishing additional collections is also valid and is used to accumulate additional nested properties within entity, relational, or interaction data types.
[0059] Referencing attribute types makes it easy to define relationships between entity types in a data model.
[0060] When a user needs an entity to use an attribute of another entity, the user can leverage reference attribute types instead of defining the attributes natively for both. For example, the L1 layer in an information model defines the use of the AffiliatedWith relationship type to link organizations and individuals. The AffiliatedWith relationship type defines the organization entity type as a reference attribute of the individual entity type. This approach to data modeling makes navigation between entities easier and allows for more refined searches.
[0061] Easier navigation between entities: In the example where organizational and individual entities are related using a relationship type, specifying the previous employer attribute for the individual entity type allows that attribute to be displayed as a hyperlink in the individual's profile panel. This allows users to easily navigate to the individual's previous employers.
[0062] Easier granular searching: Business users can more easily refine their search queries when the attributes and relationship types of the referenced entities are available for indexing (as if they were native to the referenced entities). For example, in a search of a dataset containing 100 John Smith records, entering "John Smith" in the search box will return 100 John Smith records. Adding "Acme" to the search criteria will only return John Smith records that have references containing the word "Acme" (and therefore, attributes).
[0063] Analysis attribute types are lightweight. In various embodiments, analysis attributes are not managed in the same way as other attributes when records are aggregated during a merge operation. Analysis attribute types can be used to receive and retain values delivered by the analysis solution.
[0064] When users want to make values from analytics solutions such as Reltio Insights available to business users or other applications using the Reltio REST API, they can use analytics property types. For example, if an analytics implementation calculates a customer's lifetime value and users need that value to be available to them when they view a customer profile, users can define an analytics property to retain that value and provide instructions to pass the calculation results to that property.
[0065] In a specific implementation, platform 102 assigns entity IDs (EIDs) to each data item entering the platform. Therefore, the platform can be appropriately characterized as including an EID allocation engine. Importantly, even when data is merged and / or assigned new EIDs, the lineage persistent relational database management system (RDBMS) retains the EID for each data item. Therefore, the platform can be appropriately characterized as including an old EID retention engine, whose task is to ensure that old EIDs are retained in the old EID data store when new EIDs are assigned. The old EID retention engine can be at least conceptually divided into an old EID survival sub-engine responsible for retaining all EIDs that have not been promoted to primary EIDs as old EIDs, and a lineage EID promotion sub-engine responsible for promoting the EID of the first data item merged with the second data item to the primary EID of the merged data item. The engine responsible for modifying data items can be characterized as a data item update engine, which includes merging data items and unmerging (previously merged) data items. Cross-tenant persistence also becomes possible when old EIDs are retained. In certain implementations, cross-tenant persistent EID lineage persistent RDBMS has an n-tier architecture, such as a 3-tier architecture.
[0066] Data can come from multiple sources. The processing of receiving data items can be referred to as "loading," and therefore, platform 102 can be characterized as including a new dataset loading engine. Data sources are registered, and in a particular implementation, all data ultimately loaded into a tenant will be associated with a data source. If no source is specified when creating a data item (or "object"), the source can have a default value. Therefore, the platform can be characterized as including an object registration engine that registers data items in association with their respective sources.
[0067] A crosswalk can represent a data provider or a non-data provider. The data provider supplies attribute values for objects, and these attributes are associated with the crosswalk. The non-data provider is associated with the entire entity (or relationship); it can be used to link L1 (or L2) objects to objects in other systems. Crosswalks are not necessarily limited to the entity level; each supplied attribute can be associated with a data provider crosswalk. Crosswalks are similar to primary keys or unique identifiers in the RDBMS industry.
[0068] The engine and data storage of platform 102 can be connected using a computer-readable medium (CRM). CRM is intended to represent a computer system or a network of computer systems. As used herein, "computer system" can include or be implemented as a dedicated computer system for performing the functions described herein. Typically, a computer system will include a processor, memory, non-volatile storage devices, and interfaces. A typical computer system will typically include at least a processor, memory, and means for coupling the memory to the processor (e.g., a bus). The processor can be, for example, a general-purpose central processing unit (CPU) such as a microprocessor, or a dedicated processor such as a microcontroller.
[0069] By way of example and not limitation, the memory of a computer system includes random access memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM). Memory can be local, remote, or distributed. Non-volatile storage devices are typically floppy disks or hard disks, magneto-optical disks, optical disks, read-only memory (ROM) (such as CD-ROM, EPROM, or EEPROM), magnetic cards or optical cards, or other forms of storage devices for large amounts of data. During software execution, some of this data is typically written to memory via a bus coupled to the non-volatile storage device, through direct memory access processing. Non-volatile storage devices can be local, remote, or distributed, but this is optional, as it allows for the creation of systems where all applicable data is available in memory.
[0070] Software in a computer system is typically stored in non-volatile storage. In fact, for large programs, it is not even possible to store the entire program in memory. For software to be run, it is moved, if necessary, to a computer-readable location suitable for processing, and this location is referred to as memory for illustrative purposes herein. Even when software is moved to memory for execution, the processor typically utilizes hardware registers to store values associated with the software, as well as a local cache ideally used to speed up execution. As used herein, when a software program is referred to as "implemented in a computer-readable storage medium," it is assumed that the software program is stored in an applicable, known, or convenient location (from non-volatile storage to hardware registers). The processor is considered "configured to execute the program" when at least one value associated with the program is stored in a processor-readable register.
[0071] In one example of operation, the computer system can be controlled by operating system software, which is a software program that includes a file management system, such as a disk operating system. An example of operating system software and associated file management system software is the Windows family of operating systems from Microsoft Corporation in Redmond, Washington, and its associated file management system. Other examples of operating system software and associated file management system software are the Linux operating system and its associated file management system. The file management system typically stores data in non-volatile storage and enables the processor to perform various actions required for the operating system to input and output data and store data in memory (including storing files on non-volatile storage).
[0072] A computer system's bus couples the processor to an interface. The interface facilitates coupling between devices and the computer system. The interface can be used for input and / or output (I / O) devices, modems, or networks. By way of example, and not limitation, I / O devices can include keyboards, mice or other pointing devices, disk drives, printers, scanners, and other I / O devices including display devices. By way of example, and not limitation, display devices can include cathode ray tube (CRT), liquid crystal displays (LCDs), or some other suitable, known, or convenient display devices. By way of example, and not limitation, modems can include analog modems, IDSN modems, wired modems, and other modems. By way of example, and not limitation, network interfaces can include ring network interfaces, satellite transmission interfaces (e.g., "direct PC"), or other network interfaces used to couple a first computer system to a second computer system. An interface can be considered part of a device or a computer system.
[0073] Computer systems may be compatible with, or be part of, or implemented through, a cloud-based computing system. As used herein, a cloud-based computing system is a system that provides virtualized computing resources, software, and / or information to client devices. Computing resources, software, and / or information can be virtualized by maintaining centralized services and resources accessible to edge devices through communication interfaces such as networks. "Cloud" can be a marketing term and, for the purposes of this document, may include any network described herein. Cloud-based computing systems may involve subscription-based or usage-based pricing models for services. Users can access protocols for cloud-based computing systems through web browsers or other containerized applications located on their client devices.
[0074] A computer system can be implemented as an engine, a part of an engine, or through multiple engines. As used herein, an engine includes at least two components: 1) a dedicated or shared processor or a part of such a dedicated or shared processor; and 2) hardware, firmware, and / or software modules executed by the processor. A part of one or more processors may include portions of hardware less than all the hardware including any given one or more processors, such as a subset of registers, a portion of the processor dedicated to one or more threads of a multithreaded processor, a time slice of the processor dedicated entirely or partially to executing a portion of the functionality of the engine, etc. Thus, a first engine and a second engine may have one or more dedicated processors, or the first engine and the second engine may share one or more processors with each other or with other engines. Depending on implementation-specific or other considerations, an engine may be centralized or its functionality may be distributed. An engine may include hardware, firmware, or software embodied in a computer-readable medium for execution by a processor. As described with reference to the accompanying figures, the processor uses implemented data structures and methods to transform data into new data.
[0075] The engine described herein, or the engine used by the systems and apparatus described herein, can be implemented as a cloud-based engine. As used herein, a cloud-based engine is an engine that can use a cloud-based computing system to run applications and / or functionalities. All or part of the applications and / or functionalities can be distributed across multiple computing devices and are not required to be limited to a single computing device. In some embodiments, a cloud-based engine can execute functionalities and / or modules accessible to end users through a web browser or container application without requiring the functionalities and / or modules to be locally installed on the end user's computing device.
[0076] As used herein, data storage is intended to include repositories with any applicable data organization, including tables, comma-separated value (CSV) files, traditional databases (e.g., SQL), or other applicable known or convenient organizational formats. Data storage can be implemented, for example, in physical computer-readable media on a general-purpose or special-purpose machine, in firmware, in hardware, in a combination of physical computer-readable media, firmware, and hardware, or as software embodied in applicable known or convenient devices or systems. Data storage-related components (such as database interfaces) can be considered as “part” of the data storage, part of some other system components, or a combination of the data storage and other system components; however, the physical location and other characteristics of the data storage-related components are not critical to understanding the techniques described herein.
[0077] Data storage can include data structures. As used herein, a data structure is associated with how data is stored and organized in a computer, enabling it to be used effectively within a given context. Data structures are typically based on a computer's ability to retrieve and store data at any location in its memory, specified by an address—a bit string that can itself be stored in memory and manipulated by a program. Thus, some data structures are based on calculating the address of a data item using arithmetic operations, while others are based on storing the address of the data item within the structure itself. Many data structures use both principles, sometimes combined in a non-trivial way. Implementing a data structure typically requires writing a set of procedures for creating and manipulating instances of that structure. The data storage described in this paper can be a cloud-based data storage. Cloud-based data storage is data storage compatible with cloud-based computing systems and engines.
[0078] Assuming CRM includes a network, the network can be a suitable communication network, such as the Internet or an infrastructure network. As used herein, the term "Internet" refers to a network using certain protocols (such as TCP / IP) and possibly others (such as Hypertext Transfer Protocol (HTTP) used to construct Hypertext Markup Language (HTML) documents for the World Wide Web ("web"). More generally, a network can include, for example, a Wide Area Network (WAN), a Metropolitan Area Network (MAN), a Campus Network (CAN), or a Local Area Network (LAN), but a network can have at least a suitable size in theory or be characterized in some other way (several alternatives are such as Personal Area Networks (PANs) or Home Area Networks (HANs)). Networks can include enterprise-specific networks and Virtual Private Networks (VPCs) (collectively, private networks). As the name suggests, a private network is controlled by a single entity. Private networks can include headquarters offices and optional regional offices (collectively, offices). Many offices enable remote users to connect to private network offices via some other network, such as the Internet.
[0079] Matching is a powerful functional domain and can be leveraged in various ways to support different needs. A classic scenario is matching and merging entity (profile) relationships. Within the architecture discussed in this article, relationships between linked entities can also, and often do, match and merge into a single relationship. This can happen automatically and is discussed in this article.
[0080] You can use matching to deduplicate profiles within a tenant. You can use matching on records in a file outside the tenant to identify records in that file that match profiles within the tenant. You can also use matching to match profiles stored within a data tenant with profiles within the tenant.
[0081] Figure 5 A flowchart illustrating the dynamic matching process is provided. The matching architecture is responsible for identifying profiles within a tenant that are considered semantically identical or similar. Users can use the matching configuration framework to build matching schemes. In some embodiments, users can leverage machine learning techniques to match profiles. In step 502, users can create matching rules. In step 504, users can identify attributes from the entity types they wish to use for matching. In step 506, users can write comparison formulas within each matching rule, which are responsible for the actual work of comparing one profile with other profiles. In step 508, users can map the lexical generator categories that will be responsible for creating matching candidates.
[0082] Unlike other systems, this architecture is designed for real-time operation in various embodiments. Each profile created or updated can be cleaned up instantly by a profile-level cleaner before the matching and merging processes occur. Therefore, the three-step sequence of cleanup, matching, and merging can be designed to occur in real-time at any time a profile is created or updated. This behavior makes platform 102 ideal for real-time operation within a customer ecosystem.
[0083] Finally, the survival architecture is responsible for creating the classic "golden record," but in a particular implementation, it is an on-the-fly materialized view. It is returned to any API call that retrieves the profile and contains a set of "operation values" from the profile, which are selected in real time based on survival rules defined for the entity type.
[0084] In various implementations, matching can be performed continuously and in real time. For example, when a user creates or updates a record in a tenant, the platform cleans and processes that record to find a match within the existing record set.
[0085] Each entity type (e.g., contact, organization, product) can have its own set of matching groups. In some embodiments, each matching group maintains a single rule and other properties that instruct the behavior of the rules within that group. Comparison operators (e.g., Exact, ExactOrNull, and Fuzzy) and attributes can include individual rules.
[0086] Matching terms can be used to help the matching engine quickly find candidate matching values. The comparison formula within the matching rule can be used to adjudicate candidate matching pairs and will evaluate TRUE or FALSE (or a score if the match is based on relevance).
[0087] In some embodiments, the matching function can perform one of three things with a pair of records: none (if the comparison formula determines that no match exists); issue an instruction to merge the pair; or issue an instruction to queue the pair for data administrator review. In some embodiments, the schema may include the following:
[0088] 1) Entities and relationships each have configurable attribute capabilities.
[0089] 2) The values found in the attributes are associated with cross-mappings maintained within the entity or relational object. Each profile can have multiple cross-mappings, each contributing one or more values. Data can come from multiple sources. Sources can be registered, and all data loaded into a tenant will be associated with a data source. The provided attributes can be associated with data provider cross-mappings. Cross-mappings are similar to primary keys or unique identifiers in a relational database management system (RDBMS). Cross-mappings can represent data providers or non-data providers.
[0090] 3) The data provider supplies the attribute values of the object, and the attributes are associated with cross-mapping.
[0091] 4) Non-data providers are associated with the overall entity (or relationship). In this case, it is only used to link objects in the RELTIO® system to objects in other systems. The provided attributes may not be associated with this cross-mapping.
[0092] 5) Profiles can be matched and merged, but so can relationships. While users can develop matching rules to manage profile matching and merging, relationship merging is automatic and platform-inherent. Any two relationships of the same type, each having entity A at one endpoint and entity B at the other endpoint, will be automatically merged.
[0093] 6) Attributes are inherently multi-valued, meaning they can hold multiple values. This means any attribute can collect and store multiple values from contributing sources or generated through merging with additional cross-mapping. Therefore, if a matching rule utilizes a name attribute, the matching engine will by default compare all values held within the name attribute of record A with all values held within the name attribute of record B to find matches between the values. If needed, the user can choose to match only the operation value.
[0094] 7) When two profiles are merged, the resulting profile contains an aggregation of all cross-mappings from the two contributing profiles, and therefore includes the associated attributes and values from those cross-mappings. The arrays following the attributes are also naturally merged, producing an array for each attribute that holds the aggregations of all values from the contributing attributes. Relationships benefit from the same schema and are represented in the same way as described for the merged entity. The surviving entity ID (or relationship ID) for the merged profile (or relationship) is the ID of the older of the two contributors. Beyond this, there is actually no concept of winner and loser objects.
[0095] 8) When two profiles are merged, the resulting profile contains references to all interactions previously associated with the contributing profile. (Note that interactions do not reference relationships.)
[0096] 9) If profile B is unmerged from a previous merge of A and B, B will be restored with its original entity ID. All attributes (and associated values), relationships, and interactions brought into the merged profile from profile B will be removed from the merged profile and returned to profile B.
[0097] A matchgroups construct is a collection of match groups containing the rules and operators required for correct matching. If a user needs to enable matching for a specific entity type within a tenant, they can include matchgroup sections within the entity type definition in the tenant's metadata configuration. A matchgroup section will contain one or more match groups, each containing a single rule and other elements that support that rule.
[0098] When viewing a match group in a JSON editor, users can easily see its advanced, classic elements. Rules can define Boolean formulas (see the "AND" operator used to anchor Boolean formulas in this example) to evaluate the similarity of a pair of profiles given to the match group for evaluation. Within the rule element, four other very common elements can also be held: ignoreInToken (optional), Cleanse (optional), matchTokenClasses (required), and comparatorClasses (required). The remaining visible elements (URIs, tags, etc.), along with some elements not shown in the snapshot, surround the rule and provide additional statements that affect the behavior of the group and essentially influence the rule itself.
[0099] As described below, each matching group can be specified as one of four types: Automatic, Suspicious, <Custom>, and Relevance-Based. The user-selected type allows the user to develop either Boolean or arithmetic expressions for the comparison rules. The types are described below.
[0100] Automatic type behavior: With this type set, the comparison formula is a pure Boolean, and if it evaluates to TRUE, the matching group will issue a merge instruction. Unless overridden by priority, the merge instruction will cause candidate pairs to merge.
[0101] Suspicious type behavior: In this type setting, the comparison formula is a pure Boolean, and if it evaluates to TRUE, the matching group will issue an instruction to queue for review. Unless overridden by priority, the instruction to queue for review will cause candidate pairs to appear in the "Potential Matches View" of the MDM UI.
[0102] Relevance-based type behavior: Unlike the aforementioned rules which are entirely based on Boolean constructions of rule formulas, relevance-based type expects the user to define an arithmetic scoring algorithm. The range of matching scores determines whether records are automatically merged or potential matches are created.
[0103] If a negation rule exists in the match group and evaluates to TRUE, any merge indications from other rules are downgraded to queueing for review. Therefore, automatic merging does not occur in this case. The Scope parameter of the match group defines whether the rule should be used for internal matching, external matching, or both. External matching occurs non-intrusively, and the results of the matching job are written to an output file for user review. Values for Scope are: ALL - Enables match groups for both internal and external matching (default). NONE - Disables matching for match groups. INTERNAL - Enables match groups only for matching records within the tenant. EXTERNAL - Enables match groups only for matching records from external files to records within the tenant; in certain implementations, external matching is programmatically supported via the external matching API and is available through external matching applications found in consoles such as the RELTIO® console.
[0104] If set to TRUE, only the OVs of each attribute will be used for lexicalization and comparison. For example, if the name attribute contains "Bill", "William", and "Billy", but "William" is an OV, only "William" will be cleansed, lexicalized, and compared.
[0105] Rules are the main components within a matching group. They contain the following key elements, each described in detail: ignoreInToken, cleanse, matchTokenClasses, comparatorClasses, and comparison formula.
[0106] Negation rules allow users to prevent any other rules from merging records. Matching groups can have either rules or negation rules. Negation rules have the same architecture as rules but with special behavior: if they evaluate to true, they degrade any merge instructions from other matching groups to a queue for review. It's safe to say that most matching groups across most customer configurations use rules for most matching goals. However, in some cases, it may be advantageous to additionally dedicate one or more matching groups to supporting negation rules, for the purpose of typically stopping merging based on a single condition. And when the condition is met, negation rules prevent any other rules from merging records. Therefore, in practice, a user might have seven matching groups each using rules, and an eighth group using negation rules.
[0107] Platform 102 may include a mechanism for proactively monitoring matching rules across tenants in all environments. In some embodiments, after data is loaded into a tenant, the proactive monitoring system examines each rule in the tenant over a period of time and logs its findings. Based on the percentage of entities that fail to be examined, the proactive monitoring system detects and bypasses matching rules that may cause performance issues and may notify the client. Bypassed matching rules will not participate in the matching process.
[0108] In various embodiments, the user receives a notification when the proactive monitoring system detects a matching rule that requires review. The `scoreStandalone` and `scoreIncremental` elements can be used to calculate a matching score for a profile designated as a potential match and can assist data administrators when reviewing potential matches.
[0109] Relevance-based matching is primarily designed as an alternative to strategies using automatic and suspicious rule types. With relevance-based matching, clients can create scoring algorithms of their own design. The advantage is that, in most cases, a relevance-based matching strategy can reduce complexity and the overall number of rules. This is because the instructions to merge and queue for review—which typically require separate rules (automatic and suspicious, respectively)—can often be represented by a single relevance-based rule.
[0110] A workflow is a series of sequential steps or tasks performed based on user-defined rules or conditions to execute business processes. Workflows allow users to manage complex business processes through a series of predetermined steps or tasks. Platform 102 can utilize workflows to implement processing and task management, including task assignment and tracking. Workflow processing can support creators, creation dates, deadlines, assignees, steps, and comments. In various embodiments, workflow business processes are configurable. In some embodiments, the various participants and triggers in a workflow are: participants, the people and processes involved in the workflow are participants, such as reviewers, workflow engines, hubs, and APIs; reviewers, users will be assigned the role ROLE_REVIEWER; triggers, which are used to scan activity logs to initiate scheduled processing of review workflows, for example, using a UI, where you can initiate a data change request workflow to review updates or changes to entity or profile data in your tenant. Workflow features allow users to manage business processes through a series of predetermined steps or tasks, enabling you to plan and coordinate user tasks, validate, review, and approve for multiple records.
[0111] A Data Change Request (DCR) is a collection of proposed data changes. Users without the authority to update objects, such as customer sales representatives, can suggest changes. These suggested changes are accumulated in the DCR, which is queued for review and approval by someone with approval privileges, such as a data administrator. Examples of suggested data changes include adding new attribute values, updating attribute values, deleting attribute values, and creating new objects and referenced objects. DCRs can be initiated using a web browser-based user interface for desktop or mobile applications. An example of the steps could be a user task assigned to a user to review and approve a DCR. In this example, the workflow for a DCR includes... Figure 5 The flowchart shows the sequence of the following steps.
[0112] In step 502, on the profile page in the central hub, the user can initiate DCR workflow processing in suggestion mode.
[0113] In step 504, the reviewer can approve or reject the DCR. In the DCR review panel of the UI, sub-properties within nested, referenced, or complex properties, as well as parent nested properties, are labeled with attribute values.
[0114] In step 506, if the reviewer approves the DCR, the change request is accepted using the API, and the task is marked as complete.
[0115] In alternative step 508, if the reviewer rejects the DCR, the change request is rejected using the API, and the task is marked as complete. In the inbox, you have the option to partially reject changes from the DCR. In various embodiments, the reviewer can selectively reject attributes and partially approve the DCR.
[0116] Figure 6 Depicting Figure 5 A graphical representation of the DCR workflow review process.
[0117] From the business user's perspective, workflows can be initiated for one or more profiles (manually or automatically). As the user assigning the task, the approver can review the proposed changes or enter comments.
[0118] Figure 7 These are sample data change request review panels for the UI in some embodiments. Figure 8 These are other example Data Change Request (DCR) review panels for the UI in some embodiments. As shown in these examples, in the example DCR review panel, there are labels for sub-properties within nested, referenced, or complex properties, as well as parent nested properties, that have property values.
[0119] To ensure that data administrators can make informed decisions about approving or rejecting DCRs, an additional details tab is available in the DCR review panel. Figure 9 The Additional Details tab in the DCR review panel of some embodiments is depicted. The Additional Details tab displays external information about the DCR related to the activity task stored by the user. This can be any information that can assist the data administrator during the approval process. This external information may be available in JSON format.
[0120] Partial denial can be enabled automatically for users with a deletion permission on the MDM:data.changeRequests role. Out of the box, workflow processing works with the system role ROLE_REVTEWER, which does not have this permission. Therefore, existing customers can automatically enable this feature based on the permissions they have already assigned to their data administrators (workflow reviewers). Otherwise, customers must enable partial denial by using the user management console application.
[0121] Figure 10 Interfaces for creating new characters in some embodiments are depicted. Figure 10 In this example, a new role with the exact permission (deletion) is created.
[0122] Figure 11 The interface for editing users is described. Figure 11 In this example, roles are assigned to one user / multiple users / user groups on the relevant tenant. Users can partially deny attributes in the DCR for entities and relationships. This includes nested attributes and sub-attributes of nested attributes. Furthermore, you can deny the entire DCR used to prevent the creation of new entities or relationships.
[0123] Figure 12 Interfaces for DCR in some embodiments are depicted. Figure 12 In the described interface, users can select tasks by clicking the "Tasks" tab in the Inbox and view details in the right panel. A "Reject" option appears when you hover your mouse over the task.
[0124] Figure 13 The interface for data change requests, including a "Cancel Reject" option, is described. Users can select a task by clicking the Tasks tab in the Inbox and view details in the right panel. The Reject option appears when the user hovers the mouse over the change.
[0125] In this example, users can click the reject option corresponding to the changes they want to reject. Rejected changes appear as if they are knocked out, but are not removed from the DCR until the task is approved. If a user moves to any other tab without approving the task, they may cancel all rejections. If a user chooses not to reject changes from the DCR, they can click the cancel reject button.
[0126] In some embodiments, rejection of start / end dates, roles, and tags for new entities / relationships has no effect. There may be no validation of dependencies on the rejected new entity. If there are reference attributes for that entity, the entity can continue to exist without modification.
[0127] In some embodiments, when a relationship is changed, the old relationship is removed and the new relationship is added. Therefore, by rejecting changes to a relationship, one can reject both the removal and addition of the relationship.
[0128] Figure 14 An interface for reviewing data change requests describing relationship states is depicted in some embodiments. If neither action is rejected, the following changes may occur: if the added relationship is rejected and the removed relationship is applied, the relationship may not exist; and if the added relationship is applied and the removed relationship is rejected, two relationships may exist.
[0129] Figure 15 An interface for requesting data change is depicted in some embodiments. In some embodiments, such as Figure 16 As depicted in the interface, changes to relationships and their attributes, or new or deleted relationships, can be displayed in the right panel.
[0130] If a new relationship has been added and attributes have been provided, a drop-down menu icon will appear near the relationship's title. Clicking the drop-down menu icon will display the added attributes.
[0131] Figure 16 , Figure 17 and Figure 18 It describes changes to the relationship and its attributes, or the creation or deletion of the relationship. For example... Figure 16 The depicted attributes, if already added to an existing relationship, are immediately visible, indicated by dashed lines from the relationship's header to the individual attributes. The same behavior occurs for attributes that have been modified. In some embodiments, such as... Figure 17 As depicted in the interface, if a relationship is deleted, its attributes may not be displayed. If the user changes or deletes any attributes of the relationship, those attributes are displayed similarly to other attributes. Attributes that have not been changed remain unaffected.
[0132] When a Data Change Request (DCR) is assigned to a user for review, the user can receive an email notification. When a DCR is approved or rejected, the DCR initiator can receive an email notification with the approval status, the name of the approver, and comments from the person who made the approval. Partial rejection can be automatically enabled for users with deletion permissions on the MDM:data.changeRequests role. Out-of-the-box workflow processing works with the system role ROLE_REVTEWER, which does not have this permission. Therefore, existing customers can automatically enable this feature based on the permissions they have already assigned to their data administrators (workflow reviewers). Otherwise, customers must enable partial rejection by creating a new role with the exact permissions (deletion) using the user management console application, assigning that role to one / multiple users / user groups on the relevant tenant, or performing a task action that must be assigned to your user account.
[0133] Reviewers can partially reject attributes in the DCR for entities and relationships. This includes nested attributes and sub-attributes of nested attributes. Additionally, reviewers can reject an entire DCR used to prevent the creation of new entities or relationships. To partially reject a change, you first select the task by clicking the Task in the Inbox tab and viewing the details in the right panel; the reject option appears when you hover your mouse over the change. You then click the reject option corresponding to the change the reviewer wants to reject. Rejected changes may appear as knocked out but not removed from the DCR until the task is approved. If you move to any other tab without approving the task, all rejections are canceled. If you choose not to reject changes from the DCR, click the Cancel Reject button.
[0134] In some embodiments, example limitations on the rejection attribute include that rejection has no effect on start / end dates, roles, and tags for new entities / relationships; and there is no validation of dependencies on the rejected new entity. If there are reference attributes for that entity, the entity will continue to exist without modification.
[0135] When a relationship is changed, the old relationship is removed and the new relationship is added. Therefore, while refusing to change a relationship, one can also refuse the actions of removal and addition. If neither action is refused, the following changes may occur: 1) If the added relationship is refused and the removed relationship is applied, the relationship may not exist; 2) If the added relationship is applied and the removed relationship is refused, two relationships may exist.
[0136] Changes to relationships and their attributes, or new or deleted relationships, can be displayed in the UI. In some embodiments, if a new relationship has been added and attributes have been provided, a dropdown menu icon appears near the relationship's title. Clicking the dropdown menu icon displays the added attributes. If attributes have been added to an existing relationship, they are immediately visible, indicated by dashed lines from the relationship's title to the individual attributes. The same behavior occurs for attributes that have been changed.
[0137] If a user changes or deletes any attributes of a relationship, those attributes are displayed similarly to other attributes. Attributes that have not been changed remain unaffected. If the relationship is deleted, the attributes may not be displayed.
[0138] When a Designated Review Request (DCR) is assigned to a user for review, the user can receive an email notification. When a DCR is approved or rejected, the DCR initiator receives an email notification with the approval status, the name of the approver, and comments from the person who made the approval.
[0139] Platform 102 provides the ability to centrally manage various data entities. A profile is a collection of all data associated with an entity. A profile contains the entity's attributes, relationships to the entity, and the sources of all attributes. Entity attributes may have multiple sources and multiple values. As defined by the attribute's lifecycle rules, the Operation Value (OV) is the current value for a given attribute. The profile page allows viewing and managing details of each entity within a tenant.
[0140] In various embodiments, the inbox enables users to efficiently view, manage, and process business tasks assigned to them or their teams. The inbox has filtering capabilities. Furthermore, users can create workflow tasks and take actions to review potential matches. As the assignee, you can take the desired actions on the workflow task. Platform 102 provides a simple way to review potential matches from the search view. Each workflow task can have variables associated with the overall workflow process or specific to a step. These variables typically contain internal information that can be used in customized workflows.
[0141] Users may want to access their inbox from your mobile device, such as a smartphone or tablet. Optimize the mobile experience for smaller shape factors that support gestures.
[0142] Inbox: Lists tasks and displays information such as the creator's name, task status, creation date, and due date. Task icons indicate the process to which the task belongs. More than one process can be represented in the list, and the process can vary depending on factors such as approving expense reports or matching tasks.
[0143] Team: Lists tasks assigned to team members of the user. Team members can perform any task, reassign any task, or simply view any task.
[0144] Send: Lists tasks that have been sent for approval.
[0145] All: Lists all open and closed tasks. Users with the necessary permissions will be able to access closed or resolved tasks. By default, closed tasks will be available in the inbox for a period of one year from the date they were resolved or closed.
[0146] Figure 19 A schematic diagram of an example multimodal entity parsing network system is depicted (1900). Figure 19 In the example, the multimodal entity parsing network system provides both AI-based (e.g., via a flexible entity parsing network system) and rule-based entity parsing. AI-based and rule-based entity parsing can be performed independently and / or in parallel. Figure 19 In the example, the multimodal entity parsing network system includes an AI-based matching system 1904, a rule-based matching system 1906 coupled to a computer-readable medium 1902, a multimodal matching system 1908 coupled to a computer-readable medium 1902, a large language model system coupled to a computer-readable medium 1902 (referred to separately as the large language model system 1910, collectively as the large language model system 1910), a downstream neural network classifier system coupled to a computer-readable medium 1902 (referred to separately as the downstream neural network classifier system 1912, collectively as the downstream neural network classifier system 1912), and a client system coupled to a computer-readable medium 1902 (referred to separately as the client system 1914, collectively as the client system 1914).
[0147] The AI-based matching system 1904 can be used to determine (e.g., classify) any number of entities as matches (e.g., they are the same entity) or non-matches (e.g., they are different entities) using a domain-agnostic large language model and a downstream neural network classifier. The functionality of the AI-based matching system 1904 can be provided by one or more servers and / or other computing devices. In some embodiments, the domain-agnostic large language model and / or downstream neural network classifier are located remotely from the AI-based matching system 1904, and the AI-based matching system communicates with the large language model system 1910 and / or the downstream neural network classifier system 1912 to obtain the functionality of those systems. In another embodiment, they may be part of the same system.
[0148] In some embodiments, the AI-based matching system 1904 can identify new kinds (or types) of matches that would be impossible using other types of machine learning or rule-based entity parsing systems. For example, the AI-based matching system can identify matches related to semantic distance on free-form text (e.g., product descriptions), transliteration / international format, and out-of-the-box fuzzy performance on attributes (when they perform poorly).
[0149] Examples of new types of matches that the AI-based matching system 1904 can identify are presented in the table below:
[0150] The rule-based matching system 1906 can be used to generate and / or execute matching rules to determine whether data records (e.g., data records representing entities) are the same (e.g., "match") or different (e.g., "mismatch"). In some embodiments, more specifically, the rule-based matching system 1906 can be used to determine whether an entity associated with a data record (e.g., a person, organization, and / or product, etc.) is also associated with one or more other data records. Therefore, the rule-based matching system 1906 can execute various matching rules on data records to identify duplicate data records and / or other matching data records. In some embodiments, the matching rule includes a comparison formula responsible for comparing data records to each other. In one embodiment, the comparison formula within the matching rule can be used to adjudicate candidate matching pairs and can be evaluated as TRUE or FALSE (or, if the match is based on relevance, as a score).
[0151] The multimodal matching system 1908 can be used to execute and / or manage the parallel execution of rules (e.g., matching rules) and machine learning models (e.g., matching machine learning models, such as domain-agnostic large language models, downstream neural network classifier models, etc.). For example, the multimodal matching system 1908 can manage AI-based matching systems 1904, rule-based matching systems 1906, large language model systems 1908, and / or downstream neural network classifier systems 1912 for parallel execution of machine learning-based entity parsing and rule-based entity parsing.
[0152] The large language model system 1910 can be used to execute domain-agnostic large language models. As used herein, in some embodiments, the domain-agnostic large language model (and / or other machine learning models) may include pre-trained models (e.g., incorporating zero-shot learning). The functionality of the large language model system 1910 may be performed by one or more servers and / or other computing devices. In some embodiments, the large language model system 1910 is remote from the AI-based matching system 1904, and the AI-based matching system communicates with the large language model system 1910 to obtain its functionality. In another embodiment, they may be part of the same system.
[0153] In some embodiments, the large language model system 1910 can be used to generate and / or execute an approximator network (e.g., an approximator network model). The approximator network can use a large language model (e.g., a domain-agnostic large language model) as a labeling tool for the prompt. Using the original string used here to create the prompt and the large language model as a labeling tool, the large language model system 1910 can create a smaller neural network whose purpose is to replicate the behavior of the large language model only on that single problem (e.g., without any ability to do anything else).
[0154] In some embodiments, the approximator network is used to simulate the large language model cues described above and is a fully connected neural network trained with stochastic gradient descent and standard techniques such as dropout, momentum, and / or early stopping. In some embodiments, the approximator network may require training data. A dataset with training data (such as names, etc.) can use the embeddings as is, but for cues with fewer samples, the embeddings of the approximator can be projected into a smaller dimensional space via PCA.
[0155] Here is an example:
[0156] Name 1: Rob Sylvester
[0157] Name 2: Robin Michael Sylvester
[0158] Hint: Are Rob Sylvester and Robin Michael Sylvester the same person? Answer yes or no.
[0159] The above large language models (e.g., Flan-T5-XL) have a label of 0.77.
[0160] This creates the following training example:
[0161] X: ("Rob Sylvester", "Robin Michael Sylvester")
[0162] Y: 0.77
[0163] For example, if this is done thousands of times, the system can replace a large language model (e.g., Flan-T5-XL) with this trained model.
[0164] This can be done for each cue, eliminating the need for Flan-T5-XL in our inference model and allowing us to rely solely on a set of smaller approximator networks. Fitting these approximator networks can include rich combinations of linguistic features and / or other embedding features, etc.
[0165] The downstream neural network classifier system 1912 can be used to generate a final result indicating whether entities are the same or different. More specifically, the downstream neural network classifier system 1912 can combine the outputs (e.g., feature scores) of one or more domain-agnostic large language models to generate the final result. The functionality of the downstream neural network classifier system 1912 can be performed by one or more servers and / or other computing devices. In some embodiments, the downstream neural network classifier is at the head of a self-learning curve. For example, the downstream neural network classifier can take four features and fit them to a single digit. It can be trained in seconds and is so small that it is computationally insignificant (e.g., it runs in milliseconds on a CPU). However, performance varies greatly. In some embodiments, the AI-based matching system 1904 can normalize the scores before the AI-based matching system 1904 feeds the scores into the downstream neural network classifier for several reasons.
[0166] Figure 20 A schematic diagram 2000 depicts an AI-based matching system 2002, illustrating an example of machine learning-based matching analysis and prediction using a domain-agnostic large language model and a downstream neural network classifier. The AI-based matching system 2002 can be identical to the AI-based matching system 1904. Figure 20 In the example, the AI-based matching system 2002 includes a parsing engine 2004, a machine learning-based matching engine 2006, a trimming engine 2008, an ML lexical phrase analysis engine 2010, an ML matching analysis engine 2012, an ML matching performance recommendation engine 2014, an ML matching configuration and adjustment engine 2016, an AI matching merging engine 2018, an AI-based matching system interface engine 2020, and an AI-based matching system data storage 2030.
[0167] Parsing engine 2004 is intended to denote an engine that can generate prompts for domain-agnostic large language models. More specifically, the parsing engine 2004 can identify attributes in various data models. For example, the attributes may include name attributes of name objects in individual entity models (e.g., prefix, given name, middle name, family name, suffix). In some embodiments, the parsing engine 2004 identifies the attributes and the data models based on an entity resolution request (e.g., received from a data administrator).
[0168] In some embodiments, the parsing engine 2004 may be configured to generate strings based on the attributes. For example, a first string may be a concatenation of a first set of attributes, a second string may be a concatenation of a second set of attributes, and so on. Each string may represent a specific feature for the domain-agnostic large language model.
[0169] In some embodiments, the parsing engine 2004 generates prompts for the domain-agnostic large language model based on the created strings. A single feature for the model may be the output of a prompt to the domain-agnostic large language model. The parsing engine 2004 can adjust the prompts and provide them with correct strings that are the result of parsed JSON objects representing various features, and the features can not only be accurate but also follow a very smooth distribution.
[0170] In some embodiments, the parsing engine 2004 may generate prompts for the domain-agnostic large language model executed on one or more large language model systems 1910. In one example, the prompts may include: name_one_prefix(Name_1_Prefix)="Name 1: Александр Иванов" name_two_prefix(Name_2_Prefix)="Name 2: {candidate}" name_task_suffix(Name_Task_Suffix)="Considering the many nicknames, orders, formats and alphabets used in people's names, do Name 1 and Name 2 refer to the same person? Answer Yes or No"
[0171] The "candidate" in the prompt may refer to any number of other candidate entities (e.g., 135 Western Union Avenue, Baltimore, Maryland, USA, 21201).
[0172] In one example, other prompts may include: address_one_prefix(address_one_prefix) = "135 W. Union Ave, Baltimore MD, United States, 21201" address_two_prefix(address_2_prefix) = {candidate(candidate)} `address_task_suffix(address_task_suffix)` = "Considering that addresses can be written in various formats, do address 1 and address 2 refer to the same location? Answer yes or no:"
[0173] The Machine Learning-Based Matching Engine 2006 is intended to represent an engine that determines whether any data record matches any other data record based on one or more matching machine learning models. For example, the matching machine learning model may include a domain-agnostic large language model executed on one or more large language model systems 1910 and / or a downstream neural network classifier model executed on one or more downstream neural network classifier systems 1914. The Machine Learning-Based Matching Engine 2006 can collaborate with the AI-based Matching System Interface Engine 2020 to communicate with those systems to perform such functionality.
[0174] In some implementations, the matching machine learning model includes a domain-agnostic large language model. A domain-agnostic large language model can include a large language model that has not yet been trained on domain-specific datasets and / or customer-specific datasets, etc., and can more accurately identify matches even when data records have different structures, formats, and / or information. In one example, a machine learning-based matching engine 2006 and / or a large language model can implement one or more similarity algorithms or models to determine matches. In one example, a domain-agnostic large language model can create features that measure the distance between two names, addresses, etc., and these features can be reused in any entity model, or even in attribute models related to names, addresses, etc. This reusability pattern differs from a single model that consumes all these representations of name features to output an entity score, because that model is retrained for different entity types, and the interpretability of the features is limited only to its original model.
[0175] In some embodiments, the machine learning-based matching engine 2006 can replace one or more large language models (e.g., Flan-T5 XL) with versions of Flan-T5 XL that have been fine-tuned for the specific task (e.g., approximator networks). More specifically, the machine learning-based matching engine 2006 can replace the large language model with a series of neural networks, each approximating the output of the large language model (e.g., Flan-T5 XL). These neural networks, which may be referred to as "approximator networks," are much smaller than the large language model (e.g., 1 / 1000th the size of the original large language model). The machine learning-based matching engine 2006 can be replaced before, during, or after operation.
[0176] In some embodiments, the machine learning-based matching engine 2006 can construct an approximator network model (or approximator network, or simply approximator) for each specific cue used in the large language model. In other words, if there are typically two texts X and Y, the machine learning-based matching engine 2006 can feed them into a cue that will measure the distance between X and Y, and then use the output of the large language model for that pair as the label for the downstream model. For example, this can allow training a classification head with a small amount of real-world data.
[0177] Example:
[0178] name_one_prefix(name_1_prefix) = "Full Name 1: R. Sylvester"
[0179] name_two_prefix(name_2_prefix) = "Full Name 2: Rob Michael Sylvester"
[0180] name_task_suffix(name_task_suffix) = "Considering the many nicknames, order, format, and letters used in a person's name, ...
[0181] NamePromptHelper (Name Prompt Assistant) = PromptHelper(
[0182] x1_prefix=name_one_prefix,
[0183] x2_prefix=name_two_prefix,
[0184] task_suffix=name_task_suffix )
[0186] The normalized output score of the large language model for this prompt could be 0.76, meaning the model is roughly 76% certain of being yes and 24% certain of being no. However, if the system collects many such training examples and uses them to create a training set, the machine learning-based matching engine 2006 can train a smaller, cheaper model to attempt to approximate these outputs, and then use this set of approximator networks instead of the large language model head to train the model head, reducing computational costs. (For example, if higher accuracy is required,) the machine learning-based matching engine 2006 can switch back to the large language model.
[0187] In some embodiments, the machine learning-based matching engine 2006 can use the output of a sentence transformer (e.g., the STSB multilingual sentence transformer output) to generate a vectorized representation from the original text string, which can be used as an input layer in an approximator neural network (or simply an approximator network).
[0188] The Pruning Engine 2008 is intended to represent an engine that can prune (e.g., edit) a portion of the results generated by a domain-agnostic large language model. For example, Pruning Engine 2008 can prune the results to "yes" or "no". More specifically, Pruning Engine 2008 can identify "yes" or "no" in the results and then remove the remainder of the results. This can, for example, prevent any kind of illusion and make the model execute faster.
[0189] The ML lexical phrase analysis engine 2010 is intended to represent an engine for determining the number and / or quality of identified data record matches. For example, the ML lexical phrase analysis engine 2010 can determine the number of data record matches generated by one or more matching machine learning models, and collaborate with machine learning-based matching engines 2006 to determine the quality of those matches. For example, a threshold can indicate whether too many or too few matches have been identified.
[0190] The ML Matching Analysis Engine 2012 is intended to represent an engine for determining the performance of matching machine learning models. In some embodiments, the ML Matching Analysis Engine 2012 can analyze matching machine learning models in a live multi-tenant production environment.
[0191] In some embodiments, the ML Matching Analysis Engine 2012 can generate interpretability reports to explain how model results (e.g., domain-agnostic large language model results, approximator network approximator results, machine learning model final results) were determined. Traditionally, interpretability methods for machine learning models tend to cater to data scientists as the audience, where the output is a distribution of feature coefficients, a list of feature values, or an approximation of a feature power set present in methods such as SHAP values. However, this is not helpful to other users (e.g., data administrators). Using the systems and methods described herein, the hints themselves are interpretable to data administrators. Features can query the distance between attributes or groups of attributes, but even these groupings are understandable to human readers. How different are two full names? How different are two street addresses? How different are two product descriptions? These might just happen to be large language model feature outputs. Typically, the higher the feature output, the higher the model output, and this correlation is good enough that the ML Matching Analysis Engine 2012 can output a list of these individual large language model hint outputs along with the final entity score (e.g., machine learning final results) as an interpretable version to the data administrator.
[0192] In other words, interpretability is handled by the machine learning matching analysis engine 2012 by considering the individual outputs of a large language model classifier (e.g., the Flan classifier). For example, FullName: (a score), Address: (a score). The interpretability output can map each of these to a single string. This can then be passed to a generative model to improve readability for the user.
[0193] The ML Matching Performance Recommendation Engine 2014 is intended to represent an engine for generating matching recommendation actions based on the performance of one or more machine learning models and associated matching machine learning models. For example, the ML Matching Performance Recommendation Engine 2014 may include one or more machine learning models that use (e.g., generated by a machine learning-based matching engine 2006) matching analysis to determine one or more corrective actions to improve the performance of the corresponding matching machine learning models. For example, recommendation actions may include recommendations to add, modify, and / or delete machine learning models, etc. The ML Matching Performance Recommendation Engine 2014 may also generate explanations describing the reasoning used to determine the corrective actions.
[0194] In some embodiments, the ML matching performance recommendation engine 2014 can automatically and / or in real time execute and provide recommendations. For example, the ML matching performance recommendation engine 2014 can execute continuously and / or in real time in a live production environment, generating and labeling potentially harmful matching machine learning models. The ML matching performance recommendation engine 2014 can immediately process those machine learning models and generate corresponding recommendations without any user intervention.
[0195] In some embodiments, the ML matching performance recommendation engine 2014 can be used to perform recommendation actions. The ML matching performance recommendation engine 2014 can perform recommendation actions based on (e.g., received via a graphical user interface generated by the AI-based matching system interface engine 2020) user input and / or automatically. For example, the ML matching performance recommendation engine 2014 can perform actions such as adding, modifying, and / or deleting machine learning models without requiring user input.
[0196] The ML Matching Configuration and Tuning Engine 2016 is intended to represent an engine for automatically configuring and / or tuning matching machine learning models based on user input and / or (e.g., without requiring user input). For example, the ML Matching Configuration and Tuning Engine 2016 may allow users (e.g., via a graphical user interface generated by the AI-based Matching System Interface Engine 2020) to implement reinforcement learning and / or may improve model performance based on user feedback (e.g., user input instructing whether to merge or not merge records determined to be matches).
[0197] The ML Matching Configuration and Tuning Engine 2016 can also be used to test different matching machine learning model deployment scenarios before deployment. For example, users can provide various machine learning models, and the ML Matching Configuration and Tuning Engine 2016 can simulate how matching performance can be improved or degraded based on the tested scenario. For example, this can reduce the computational impact of deploying harmful machine learning models.
[0198] In some embodiments, the ML matching configuration and tuning engine 2016 can configure and / or tune a matching machine learning model based on user input received in response to prompts generated by the ML matching configuration and tuning engine 2016. For example, the matching rule configuration and tuning engine 1914 can prompt the user with various questions, such as "Are John and Johnny the same person?" and the matching rule configuration and tuning engine 1914 can configure and tune one or more matching machine learning models based on the responses.
[0199] AI Match Merge Engine 2018 is intended to represent an engine that can merge (e.g., two or more entities') two or more matching data records. For example, AI Match Merge Engine 2018 can merge a first data record with a second data record and retain the second data record while ignoring (e.g., deleting, ignoring) the first data record in any subsequent operation. In other examples, Merge Engine 1996 can create a new data record from a first data record and a second data record and ignore both the first and second data records in any subsequent operation.
[0200] In some embodiments, the AI matching merging engine 2018 can be used to identify candidate data records for potential match identification. More specifically, the merging engine 1916 can identify various data records (e.g., data records in a live multi-tenant enterprise environment). Each data record can be associated with an entity (e.g., an individual, organization, enterprise, product), and each data record can include various record fields (e.g., first name, last name, social security number, email address, phone number, city, state, county, postal code, area code, country, and organization, etc.) and corresponding record field values (e.g., John, Doe, 555-55-5555, john.doe@domain.com, 555-555-5555, Boston, MA, Suffolk, 02109, 617, USA, and Acme, etc.). The AI matching merging engine 2018 can identify candidate records with the same corresponding field values, as well as records with different values, formats, and structures, etc. Candidate records (e.g., machine learning-based matching engine 2006) can be used to determine matches between data records.
[0201] The AI-based matching system interface engine 2020 is intended to represent an engine capable of sending requests, transmitting and receiving communications, and / or otherwise providing communications to one or more of the systems, engines, devices, and / or data stores described herein. In a particular implementation, the AI-based matching system interface engine 2020 can be used to encrypt and decrypt communications. The AI-based matching system interface engine 2020 can be used to send requests to one or more systems and receive data from one or more systems via a network or a portion of a network (e.g., computer-readable medium 102). In a particular implementation, the AI-based matching system interface engine 2020 can send requests and receive data via a connection, which may be wholly or partially wireless. The AI-based matching system interface engine 2020 can request and receive messages and / or other communications from associated systems and / or engines. Communications may be stored in data store 230.
[0202] In one example, the AI-based matching system interface engine 2020 can communicate with large language models and downstream neural network classifier systems to perform the functionality described in this paper.
[0203] In some embodiments, the AI-based matching system interface engine 2020 can be used to present visual, audio, and / or tactile information. In some embodiments, the AI-based matching system interface engine 2020 generates graphical user interface components (e.g., server-side graphical user interface components) that can be rendered as a complete graphical user interface on various systems (e.g., client systems). The AI-based matching system interface engine 2020 can be used to present an interactive graphical user interface for displaying and receiving information.
[0204] In some embodiments, the AI-based matching system interface engine 2020 can be used to present graphical user interface elements that represent the graphical user interface. More specifically, the AI-based matching system interface engine 2020 can generate graphical user interface elements that indicate the type of processing used to determine a match. For example, one graphical user interface element might indicate that the match was determined using matching rules, while other graphical user interface elements might indicate that the match was determined using machine learning. This can give users more confidence when determining whether to merge records. For example, having two indicators can increase the likelihood of data record matching.
[0205] Figure 21 A flowchart 2100 depicts an example method for flexible entity resolution based on AI. In this flowchart, the diagram of the process and / or the sequence diagram, and other flowcharts, diagrams of other processes and / or other sequence diagrams, the flowcharts exemplify the sequence of modules by way of example. It should be understood that modules can be reorganized for parallel execution or reordered where applicable. Furthermore, for clarity, some modules that have been included may have been removed to avoid providing excessive information, and some modules that have been included may have been removed but may have been included for clarity.
[0206] In module 2102, an AI-based matching system (e.g., AI-based matching system 2002) receives an entity resolution request. The entity resolution request may indicate a first entity (e.g., “Александр Иванов” at the address “135 West Union Avenue, Baltimore, 21201, USA”) and a second entity (e.g., “Dr. Sarah Jackson” at the address “123 West Union Avenue, Baltimore, 99801, USA”). In some embodiments, an AI-based matching system interface engine (e.g., AI-based matching system interface engine 2020) receives the entity resolution request via a network (e.g., computer-readable medium 1902) (e.g., from client system 1914). For example, a data administrator's client system 1904 may provide a request to help determine whether entities are the same or different. In some embodiments, the entity resolution request may indicate any number of second entities (e.g., making the first entity comparable to any number of other entities). For example, the second entities may include any number of candidate entities.
[0207] In module 2104, an AI-based matching system identifies multiple first attributes in the data model. For example, the first attributes may include the name attributes (e.g., prefix, first name, middle name, last name, suffix) of the name object of the individual entity model. In some embodiments, a syntactic analysis engine (e.g., syntactic analysis engine 2004) (e.g., based on an entity syntactic analysis request) identifies the first attributes and the data model.
[0208] In module 2106, an AI-based matching system identifies multiple second attributes in the data model. For example, the second attributes may include attributes of address objects in the individual entity model (e.g., address line 1, address line 2, city, etc.). In some embodiments, a syntactic analysis engine (e.g., based on entity parsing requests) identifies the second attributes.
[0209] In module 2108, the AI-based matching system creates a first string based on multiple first attributes of the data model. For example, the syntactic analysis engine can concatenate the first attributes in sequence.
[0210] In module 2110, the AI-based matching system creates a second string based on multiple second attributes of the data model. For example, the syntactic analysis engine can concatenate the second attributes sequentially.
[0211] In module 2112, the AI-based matching system generates a first prompt based on a first character string. In some embodiments, the machine learning-based matching engine (e.g., the machine learning-based matching engine 2006) generates the first prompt. For example, the machine learning-based matching engine may generate the first prompt for a large language model executed on one or more large language model systems (e.g., the large language model system 1910). The large language model system may be separate from the AI-based matching system. For example, the large language model system may be executed on one or more apparatuses, and the AI-based matching system may be executed on one or more other remote apparatuses. In one example, the first prompt may include: name_one_prefix(Name_1_Prefix)="Name 1: Александр Иванов" name_two_prefix(Name_2_Prefix)="Name 2: {candidate}" name_task_suffix(Name_Task_Suffix)="Considering the many nicknames, orders, formats and alphabets used in personal names, do Name 1 and Name 2 refer to the same person? Answer Yes or No"
[0212] The "candidate" in the prompt may refer to any number of other candidate entities (e.g., 135 Western Union Avenue, Baltimore, Maryland, USA, 21201).
[0213] In module 2114, the AI-based matching system generates a second prompt based on a second character string. For example, the machine learning-based matching engine may generate the second prompt for a large language model executed on one or more large language model systems. In one example, the second prompt may include: address_one_prefix(Address_1_Prefix)="135 W. Union Ave, Baltimore MD, United States, 21201)" address_two_prefix(Address_2_Prefix)={candidate} address_task_suffix(Address_Task_Suffix)="Considering that addresses can be written in multiple formats, do Address 1 and Address 2 refer to the same location? Answer Yes or No:"
[0214] In module 2116, the AI-based matching system provides initial hints to the domain-agnostic large language model. For example, the domain-agnostic large language model could be a Flan-T5 XL model that has not yet been trained on any domain-specific dataset or customer dataset, etc. In some embodiments, the AI-based matching system interface engine provides initial hints to the domain-agnostic large language model.
[0215] In module 2118, the large language model system generates a first domain-agnostic large language model result using a first prompt through a domain-agnostic large language model. For example, the result may include natural language results such as "Yes, it is a match..." or "No, it is not a match." In some embodiments, the result may include feature scores (e.g., full name: 0.75, address lines 1 & 2: 0.63, address city: 0.94, and name list: 0.47).
[0216] In module 2120, an AI-based matching system prunes the results of a large language model that is unknown in the first domain. For example, the AI-based matching system can prune the results to "yes" or "no". For example, it can identify yes or no and remove the remainder of the result. This can, for example, prevent any kind of illusion. In some embodiments, a pruning engine (e.g., pruning engine 2008) prunes the results.
[0217] In module 2122, the AI-based matching system provides a second hint to the domain-agnostic large language model. In some embodiments, the AI-based matching system interface engine provides the second hint to the domain-agnostic large language model.
[0218] In module 2124, the AI-based matching system uses a second prompt to generate a second domain-agnostic large language model result. For example, the result may include a natural language result such as "Yes, it is a match..." or "No, it is not a match." In some embodiments, the result may include feature scores (e.g., full name: 0.75, address lines 1 & 2: 0.63, address city: 0.94, and name list: 0.47).
[0219] In module 2126, an AI-based matching system prunes the results of a second-domain-agnostic large language model. For example, the AI-based matching system can prune the results to "yes" or "no". For instance, it can identify yes or no and remove the remainder of the result. This can, for example, prevent any kind of illusion. In some embodiments, the pruning engine prunes the results.
[0220] In module 2128, the AI-based matching system generates a final machine learning result through a downstream neural network classifier based on the pruned results of a first-domain-agnostic large language model and a second-domain-agnostic large language model. For example, the final output can be a score based on the first-domain-agnostic and second-domain-agnostic large language model results. In some embodiments, the downstream neural network classifier can combine the feature scores of the first-domain-agnostic and second-domain-agnostic large language model results.
[0221] In module 2130, the AI-based matching system merges the first entity and the second entity based on the final result of machine learning. In some embodiments, the AI matching merging engine (e.g., AI Matching Merging Engine 2018) merges the first entity and the second entity because, based on the final result, they are considered to be the same entity. For example, a data administrator can review the final result and determine that they are the same entity. In other examples, the AI-based matching system can automatically determine that they are the same entity (e.g., based on a comparison with a threshold score).
[0222] Figure 22 A flowchart 2200 depicts an example method for interactive parallelized multimodal matching. In this flowchart, the diagram of the process and / or the sequence diagram, and other flowcharts, diagrams of other processes and / or other sequence diagrams, the flowcharts exemplify the sequence of modules by way of example. It should be understood that modules can be reorganized for parallel execution, or reordered where applicable. Furthermore, for clarity, some modules may have been removed to avoid providing excessive information, and some included modules may have been removed but may have been included for clarity.
[0223] In module 2202, the multimodal matching system identifies at least two distinct data records from a plurality of distinct data records. Each data record may be associated with a corresponding entity, and each data record may include multiple corresponding record fields and corresponding record field values. At least the first record field value of the first data record differs from the corresponding first record field value of the second data record.
[0224] In module 2204, the multimodal matching system determines, based on multiple different matching rules, whether the corresponding entity associated with the first data record and the corresponding entity associated with the second data record include the same entity.
[0225] In module 2206, the multimodal matching system determines, in parallel with rule-based determination, whether the corresponding entity associated with the first data record and the corresponding entity associated with the second data record include the same entity, based on one or more machine learning models.
[0226] In module 2208, the multimodal matching system, in response to a rule-based determination indicating that the corresponding entities are the same entity, presents a first graphical user interface element of the graphical user interface.
[0227] In module 2210, the multimodal matching system presents a second graphical user interface element that indicates, based on machine learning, whether the corresponding entities are the same entity or not.
[0228] In module 2212, the multimodal matching system receives user input through a graphical user interface.
[0229] In module 2214, the multimodal matching system merges the first data record and the second data record based on user input.
[0230] Figure 23 A flowchart 2300 depicts an example method for analyzing matching rules and generating machine learning-based matching rule recommendation actions. In this flowchart, the diagram of the process and / or the sequence diagram, and other flowcharts, diagrams of other processes and / or other sequence diagrams, the flowcharts exemplify the sequence of modules by way of example. It should be understood that modules can be reorganized for parallel execution or reordered where applicable. Furthermore, for clarity, some included modules may have been removed to avoid providing excessive information, and some included modules may have been removed but may have been included for clarity.
[0231] In module 2302, the multimodal matching system identifies one or more of a plurality of different matching rules. Each matching rule can be configured to identify whether at least two different data records from a plurality of different data records (e.g., data records in a big data enterprise environment) are each associated with the same entity (e.g., a person, organization, and / or product). The plurality of different data records can be deployed in a production environment (e.g., live data in a production enterprise environment).
[0232] In module 2304, the multimodal matching system executes one or more matching rules on multiple different data records.
[0233] In module 2306, the multimodal matching system determines the respective performance for one or more matching rules.
[0234] In module 2308, the multimodal matching system generates matching rule recommendation actions based on the performance of one or more machine learning models and one or more matching rules.
[0235] In module 2310, the multimodal matching system performs the matching rule recommendation action.
[0236] Figure 24 A flowchart 2400 illustrates an example method for flexible AI-based entity resolution using one or more approximator networks. In this flowchart, the diagram of the process and / or the sequence diagram, and other flowcharts, diagrams of other processes and / or other sequence diagrams, the flowcharts exemplify the sequence of modules by way of example. It should be understood that modules can be reorganized for parallel execution or reordered where applicable. Furthermore, for clarity, some modules may have been removed to avoid providing excessive information, and some included modules may have been removed but may have been included for clarity.
[0237] In module 2402, an AI-based matching system (e.g., AI-based matching system 2002) receives an entity resolution request. The entity resolution request may indicate a first entity (e.g., “Александр Иванов” at the address “135 West Union Avenue, Baltimore, 21201, USA”) and a second entity (e.g., “Dr. Sarah Jackson” at the address “123 West Union Avenue, Baltimore, 99801, USA”). In some embodiments, an AI-based matching system interface engine (e.g., AI-based matching system interface engine 2020) receives the entity resolution request via a network (e.g., computer-readable medium 1902) (e.g., from client system 1914). For example, a data administrator's client system 1904 may provide a request to help determine whether entities are the same or different. In some embodiments, the entity resolution request may indicate any number of second entities (e.g., allowing the first entity to be compared with any number of other entities). For example, the second entities may include any number of candidate entities.
[0238] In module 2404, an AI-based matching system identifies multiple first attributes in the data model. For example, the first attributes may include the name attributes (e.g., prefix, first name, middle name, last name, suffix) of the name object of the individual entity model. In some embodiments, a syntactic analysis engine (e.g., syntactic analysis engine 2004) (e.g., based on an entity syntactic analysis request) identifies the first attributes and the data model.
[0239] In module 2406, an AI-based matching system identifies multiple second attributes in the data model. For example, the second attributes may include attributes of address objects in the individual entity model (e.g., address line 1, address line 2, city, etc.). In some embodiments, a syntactic analysis engine (e.g., based on entity parsing requests) identifies the second attributes.
[0240] In module 2408, the AI-based matching system creates a first string based on a plurality of first attributes of the data model. For example, the syntax analysis engine may concatenate the first attributes in sequence.
[0241] In module 2410, the AI-based matching system creates a second string based on a plurality of second attributes of the data model. For example, the syntax analysis engine may concatenate the second attributes in sequence.
[0242] In module 2412, the AI-based matching system generates a first prompt based on the first string. In some embodiments, the machine learning-based matching engine (e.g., the machine learning-based matching engine 2006) generates the first prompt. For example, the machine learning-based matching engine may generate the first prompt for a large language model executed on one or more large language model systems (e.g., the large language model system 1910). The large language model system may be separate from the AI-based matching system. For example, the large language model system may be executed on one or more devices, and the AI-based matching system may be executed on one or more other remote devices. In one example, the first prompt may include: name_one_prefix(Name_1_Prefix)="Name 1: Александр Иванов" name_two_prefix(Name_2_Prefix)="Name 2: {candidate}" name_task_suffix(Name_Task_Suffix)="Considering the many nicknames, orders, formats and alphabets used in people's names, do Name 1 and Name 2 refer to the same person? Answer Yes or No"
[0243] The "candidate" in the prompt may refer to any number of other candidate entities (e.g., 135 Western Union Avenue, Baltimore, Maryland, USA, 21201).
[0244] In module 2414, the AI-based matching system generates a second prompt based on the second string. For example, the machine learning-based matching engine may generate the second prompt for a large language model executed on one or more large language model systems. In one example, the second prompt may include: address_one_prefix(Address_1_Prefix)="135 W. Union Ave, Baltimore MD, United States, 21201)" address_two_prefix(address_2_prefix) = {candidate(candidate)} `address_task_suffix(address_task_suffix)` = "Considering that addresses can be written in various formats, do address 1 and address 2 refer to the same location? Answer yes or no:"
[0245] In module 2416, the AI-based matching system provides a first suggestion to the approximator network (e.g., performed by one or more large language model systems 1910). For example, the approximator network may be a Flan-T5 XL model that has not yet been trained on any domain-specific dataset or customer dataset, etc. In some embodiments, the AI-based matching system interface engine provides the first suggestion to the approximator network.
[0246] In module 2418, the large language model system generates a first approximator network result using the first prompt through the approximator network. For example, the result may include natural language results such as "Yes, it is a match..." or "No, it is not a match." In some embodiments, the result may include feature scores (e.g., full name: 0.75, address lines 1 & 2: 0.63, address city: 0.94, and name list: 0.47).
[0247] In module 2420, an AI-based matching system trims the results of the first approximator network. For example, the AI-based matching system can trim the results to "yes" or "no". For example, it can identify yes or no and remove the remainder of the result. This can, for example, prevent any kind of illusion. In some embodiments, a trimming engine (e.g., trimming engine 2008) trims the results.
[0248] In module 2422, the AI-based matching system provides a second cue to the approximator network. In some embodiments, the AI-based matching system interface engine provides the second cue to the approximator network.
[0249] In module 2424, the AI-based matching system generates a second approximator network result using a second prompt through the approximator network. For example, the result may include a natural language result such as "Yes, it is a match..." or "No, it is not a match." In some embodiments, the result may include feature scores (e.g., full name: 0.75, address lines 1 & 2: 0.63, address city: 0.94, and name list: 0.47).
[0250] In module 2426, an AI-based matching system trims the results of the second approximator network. For example, the AI-based matching system can trim the results to "yes" or "no". For instance, it can identify yes or no and remove the remainder of the result. This can, for example, prevent any type of illusion. In some embodiments, the trimming engine trims the results.
[0251] In module 2428, the AI-based matching system generates a final machine learning result through a downstream neural network classifier based on the pruned results of the first and second approximator networks. For example, the final output may be a score based on the results of the first and second language models. In some embodiments, the downstream neural network classifier may combine the feature scores of the first and second language model results.
[0252] In module 2430, the AI-based matching system merges the first entity and the second entity based on the final result of machine learning. In some embodiments, the AI matching merging engine (e.g., AI Matching Merging Engine 2018) merges the first entity and the second entity because, based on the final result, they are considered to be the same entity. For example, a data administrator can review the final result and determine that they are the same entity. In other examples, the AI-based matching system can automatically determine that they are the same entity (e.g., based on a comparison with a threshold score).
Claims
1. A system comprising: One or more processors; as well as A memory storing instructions that, when executed by one or more processors, cause the system to perform the following: Receive an entity resolution request, wherein the entity resolution request indicates a first entity and a second entity; Identify multiple primary attributes in the data model; Identify multiple second attributes in the data model; A first string is created based on the multiple first attributes of the data model; A second string is created based on the multiple second attributes of the data model; A first prompt is generated based on the first string; A second prompt is generated based on the second string; Provide the first hint to the domain-agnostic large language model; The first prompt is used to generate the first domain-agnostic large language model result through the domain-agnostic large language model; Prune the results of the large language model that is unknown in the first domain; Provide the second hint to the domain-agnostic large language model; The second hint is used to generate a second domain-agnostic large language model result through the domain-agnostic large language model; Prune the results of the large language model that is unknown in the second domain; The downstream neural network classifier generates the final machine learning result based on the results of the first-domain unknown large language model and the second-domain unknown large language model after pruning. The first entity and the second entity are merged based on the final result of the machine learning.
2. A method comprising: Receive an entity resolution request, wherein the entity resolution request indicates a first entity and a second entity; Identify multiple primary attributes in a data model; Identify multiple second attributes in the data model; A first string is created based on the multiple first attributes of the data model; A second string is created based on the multiple second attributes of the data model; A first prompt is generated based on the first string; A second prompt is generated based on the second string; Provide the first hint to the domain-agnostic large language model; The first prompt is used to generate the first domain-agnostic large language model result through the domain-agnostic large language model; Prune the results of the large language model that is unknown in the first domain; Provide the second hint to the domain-agnostic large language model; The second hint is used to generate a second domain-agnostic large language model result through the domain-agnostic large language model; Prune the results of the large language model that is unknown in the second domain; The downstream neural network classifier generates the final machine learning result based on the results of the first-domain unknown large language model and the second-domain unknown large language model after pruning. The first entity and the second entity are merged based on the final result of the machine learning.
3. A system comprising: One or more processors; as well as A memory storing instructions that, when executed by one or more processors, cause the system to perform the following: Receive an entity resolution request, wherein the entity resolution request indicates a first entity and a second entity; Identify multiple primary attributes in a data model; Identify multiple second attributes in the data model; A first string is created based on the multiple first attributes of the data model; A second string is created based on the multiple second attributes of the data model; A first prompt is generated based on the first string; A second prompt is generated based on the second string; Provide the first hint to the approximator network; The first approximator network result is generated using the first prompt through the approximator network; Prune the results of the first approximator network; Provide the second hint to other approximator networks; The second approximator network result is generated using the second prompt through the other approximator networks; Prune the results of the second approximator network; The downstream neural network classifier generates the final machine learning result based on the results of the first and second approximator networks after pruning. The first entity and the second entity are merged based on the final result of the machine learning.
4. A method comprising: Receive an entity resolution request, wherein the entity resolution request indicates a first entity and a second entity; Identify multiple primary attributes in a data model; Identify multiple second attributes in the data model; A first string is created based on the multiple first attributes of the data model; A second string is created based on the multiple second attributes of the data model; A first prompt is generated based on the first string; A second prompt is generated based on the second string; Provide the first hint to the approximator network; The first approximator network result is generated using the first prompt through the approximator network; Prune the results of the first approximator network; Provide the second hint to other approximator networks; The second approximator network result is generated using the second prompt through the other approximator networks; Prune the results of the second approximator network; The downstream neural network classifier generates the final machine learning result based on the results of the first and second approximator networks after pruning. The first entity and the second entity are merged based on the final result of the machine learning.
