Adaptive Event Processing System for Heterogeneous Schema Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise environments face challenges in harnessing insights from diverse applications due to information gaps, as big data analytics systems require domain-specific data models and processing models that are tightly tied to the enterprise's data schema, making it difficult to accommodate new data schemas without extensive reprogramming.
Innovation Solution
An adaptive event processing system with a domain-agnostic service that uses an adaptive data model and adaptive processing model, supporting runtime event type discovery, event schema management, and configurable partitioning of events by type and time, allowing for the processing of heterogeneous events across multiple schemas without predefined schemas or analysis rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If domain-specific data models and processing models are used with upfront dependencies tied to enterprise data schemas, then analysis precision is improved, but adaptability deteriorates making it difficult to accommodate new data schemas without extensive reprogramming
Solution Approach 1:
The patent implements dynamic schema evolution by allowing the system to automatically discover and adapt to new event schemas at runtime. The adaptive processing model detects new event types and schemas without requiring predefined models, enabling the system to dynamically adjust its data models and processing logic to accommodate changing enterprise data schemas while maintaining analysis precision through reactive schema registration and model updates.
2Adaptability or versatility
If heterogeneous events from multiple applications with different formats and schemas are processed, then versatility is improved, but device complexity increases making it difficult to bridge information gaps
Solution Approach 1:
The patent introduces an adaptive event processing system as an intermediary layer between heterogeneous enterprise applications and analytics systems. This mediator automatically discovers event schemas, registers new event types, and transforms diverse event formats into a unified processing model, thereby enabling the system to handle heterogeneous events from multiple applications without requiring complex custom integration logic for each data source.
3Adaptability or versatility
If runtime event type discovery and schema management are implemented, then adaptability is improved, but loss of information increases due to the need to manage dynamic schemas
Solution Approach 1:
The patent implements self-service schema management where the adaptive processing model automatically discovers event schemas from incoming events, infers data structures, and registers new schemas without requiring external schema definitions or manual configuration. The system autonomously manages schema evolution by detecting new event types, extracting schema information from event payloads, and updating its internal data models, thereby maintaining adaptability while minimizing information loss through automated schema capture.
Data Source
AI summary
An event processing system comprising a processor and a non-transitory computer readable medium coupled to the processor. The non-transitory computer readable medium stores computer-readable code executable to provide a domain-agnostic service for analyzing data and processing heterogeneous events conforming to a plurality of schemas using the service. The domain-agnostic service comprises an adaptive data model and an adaptive processing model. The adaptive data model supports runtime event type discovery and event schema management that reacts to new schemas discovered at runtime. The adaptive processing model supports configurable partitioning of events by event type and time and is reactive to create new aggregations and analyses based on the propagation of new partitions.