LLM Explanations for Entity Matching Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ML-based entity matching systems provide black-box predictions that are difficult for users to understand, as the explanations are not human-readable and intuitive, particularly in semi-structured data contexts like matching bank statements to invoices.
Innovation Solution
Implementations utilize large language models (LLMs) to generate conversational-style explanations for entity matches by converting semi-structured data into natural language prompts, leveraging models like ChatGPT to provide understandable reasons for query-target predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ML-based entity matching systems are used to provide predictions, then matching accuracy is improved, but the explanations become difficult for users to understand
Solution Approach 1:
The patent introduces a large language model as an intermediary component between the ML-based entity matching system and the user. This mediator translates the black-box predictions into human-readable conversational explanations, allowing the system to maintain high matching accuracy while improving user understanding through natural language descriptions of the matching rationale
Solution Approach 2:
The system segments the explanation generation process into distinct components: the ML model generates predictions, the large language model processes these predictions along with query and target entities, and finally produces structured conversational explanations. This segmentation allows each component to specialize in its function while collectively resolving the contradiction between accuracy and understandability
2Productivity
If black-box ML predictions are provided, then processing efficiency is improved, but the explanations are not human-readable and intuitive
Solution Approach 1:
The system performs preliminary action by having the large language model generate conversational explanations immediately after the ML model produces predictions, while the query and target entities are still available in context. This timing allows the explanation generator to access all necessary information without requiring additional data retrieval, maintaining processing efficiency while preventing information loss
Solution Approach 2:
The patent merges multiple functions into a unified pipeline: entity matching, prediction generation, and explanation creation are combined in a single processing flow. The large language model simultaneously processes the query entity, target entity, and ML prediction to generate comprehensive explanations, thereby maintaining efficiency while preserving explanation quality
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving, by an entity matching ML model, a query and target pair including a query entity and a target entity, providing, by the entity matching ML model, a query-target prediction by processing the query entity and the target entity, the query-target prediction indicating a match type between the query entity and the target entity, generating a prompt by populating a prompt template with at least a portion of the query-target prediction, inputting the prompt into a large language model (LLM), and receiving, from the LLM, an explanation that is responsive to the prompt and that describes one or more reasons for the query-target prediction output by the entity matching ML model.