Entity Database System for Real-Time and Batch Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database technologies lack support for entity identification, canonicalization, persistence, and querying, making it difficult for enterprises to manage and resolve entities from multiple sources effectively.
Innovation Solution
An entity database system with components for real-time and high-throughput entity resolution, canonicalization, persistence, and querying, utilizing a declarative language to efficiently manage and standardize entity representations, and ensure consistent entity states across different data stores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If current database technologies (relational, document-oriented, key-value) are used, then data storage and retrieval are supported, but entity identification, canonicalization, persistence, and querying capabilities are lacking
Solution Approach 1:
The patent creates a universal entity database system that integrates multiple database functionalities (storage, retrieval, entity identification, canonicalization, persistence, and querying) into a single system. The entity resolution engine serves as a multi-functional component that handles various entity management tasks across different data sources, eliminating the need for separate systems for each function.
Solution Approach 2:
The entity resolution engine acts as an intermediary layer between traditional database systems and entity management requirements. It receives data from multiple sources, performs entity resolution and canonicalization, and then stores the processed data in the entity database, mediating between raw data and structured entity information.
2Reliability
If entities are resolved and canonicalized from multiple data sources, then data integrity and consistency are improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary entity resolution and canonicalization actions when data is first ingested from multiple sources. By pre-processing the data and establishing entity identities upfront, the system avoids the need for complex real-time resolution during querying, reducing subsequent processing time while maintaining data integrity.
Solution Approach 2:
The entity resolution process is segmented into distinct components: data ingestion, entity resolution, canonicalization, and storage. This segmentation allows each component to be optimized independently, with the entity resolution engine focusing on identification and matching while the database handles storage and retrieval, improving overall processing efficiency.
3Adaptability or versatility
If a comprehensive entity database system is implemented, then entity management capabilities are enhanced, but system complexity and implementation difficulty increase
Solution Approach 1:
The entity database system implements self-service capabilities through automated entity resolution and canonicalization. The entity resolution engine automatically identifies and merges entities from multiple sources without requiring manual intervention, and the system self-manages the canonicalization process, reducing implementation and operational complexity.
Solution Approach 2:
The entity resolution engine serves as an intermediary that abstracts the complexity of multi-source entity management. It provides a standardized interface for entity operations, shielding users from the underlying complexity of resolving entities across different data sources and formats.
4Speed
If real-time entity resolution is performed on data streams, then responsiveness is improved, but processing throughput decreases
Solution Approach 1:
The system dynamically adjusts its processing mode based on requirements. For time-critical operations, it performs real-time entity resolution on incoming data streams with optimized performance. For batch operations, it switches to high-throughput processing mode, allowing the system to maintain both responsiveness and productivity in different operational contexts.
Solution Approach 2:
The processing architecture is segmented into real-time processing components for immediate response and batch processing components for high throughput. This segmentation allows the system to handle different types of workloads simultaneously, maintaining fast response times for critical operations while achieving high overall processing capacity through batch operations.
Data Source
AI summary
A computerized method for implementing a matching platform for entities includes a real-time processing layer using real-time linking on an input data stream, and a high-throughput layer linking entities from a batch data source stream. The real-time linking results are stored in a state change store and the high-throughput linking results are stored in a state store. On-demand linking uses the state change store and the state store. A lambda switch ensures that the entity states remain consistent when the on-demand linking switches from an older generation of the state store and the stage change store to a newer generation of the state store and the state change store. These stores are implemented in an entity database that parses queries into instructions and creates an instruction stream to control entity matching.