Vendor Name Embedding Graphs for Duplicate Record Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inconsistent vendor naming in databases leads to erroneous treatment of a single vendor as multiple entities, hampering efficient data management and analysis, particularly in bookkeeping software systems.
Innovation Solution
A two-fold graphical embedding process using a large language model (LLM) to transform vendor names into embeddings, create initial entity graphs based on similarity, and recalibrate edge weights with transactional data to merge duplicate vendor entries into a single entity representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional vendor naming methods are used, then data entry is simple, but vendor identification accuracy deteriorates
Solution Approach 1:
The patent introduces an intermediary embedding layer that transforms vendor names into vector representations. This embedding layer acts as a mediator between the raw vendor names and the similarity comparison process, enabling accurate identification of duplicate vendors despite naming variations without requiring complex rule-based matching systems.
Solution Approach 2:
The patent replaces traditional mechanical string-matching mechanisms with a neural embedding-based similarity measurement system. Instead of relying on exact string matches or simple fuzzy matching algorithms, the system uses neural networks to create embeddings and compute cosine similarities, providing more robust and accurate vendor identification.
2Reliability
If multiple vendor names are treated as separate entities, then data storage is straightforward, but data analysis reliability deteriorates
Solution Approach 1:
The patent merges multiple vendor names that refer to the same entity into a single unified vendor representation. By using embedding-based similarity clustering, the system identifies and consolidates duplicate vendor records, ensuring that all transactions with the same vendor are aggregated under a single entity, thereby improving data analysis reliability and management efficiency.
Solution Approach 2:
The patent creates a universal vendor representation system that can handle multiple naming conventions and contexts. The embedding model learns to represent vendors in a way that captures their essential identity across different names, allowing the system to universally identify and track vendors regardless of how they are referred to in different datasets or contexts.
3Measurement precision
If simple name matching is used, then processing speed is high, but vendor deduplication accuracy deteriorates
Solution Approach 1:
The patent performs preliminary embedding generation for vendor names before the actual deduplication process. By pre-computing and storing the embeddings in a database, the system prepares the data in advance, so that during the deduplication process, it only needs to compute similarities between pre-generated embeddings rather than processing raw names from scratch, thereby reducing processing time while maintaining high accuracy.
Solution Approach 2:
The patent changes the parameter space from raw string names to embedded vector representations. This parameter transformation allows the system to use efficient similarity metrics like cosine similarity on high-dimensional vectors, which can be computed quickly using optimized algorithms, achieving both high accuracy and fast processing speeds.
Data Source
AI summary
A system for vendor deduplication. The system creates embeddings of entity names and creates an initial entity graph comprising entities whose embeddings are related. The initial entity graph includes nodes representing the entity names linked together by edge weights indicating their similarity. The system adjusts the edge weights between the entity names according to transactional data related to the entities to create a final entity graph and merges the entity names as a common entity based on the adjusted edge weights in the final entity graph.