Stream Profiling for Complex Event Processing Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex Event Processing (CEP) developers face challenges in defining business logic for analyzing event streams due to unknown characteristics, lack of persistent data, and difficulty in examining stream history, making it hard to perform iterative analysis and post-hoc analysis.
Innovation Solution
A stream profiling system that generates event-based and query-based profiles by analyzing input events and query registration events, using a CEP engine with processing resources to provide statistical and data mining characteristics, and storing these profiles for deeper understanding and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If CEP developers attempt to analyze event streams with unknown characteristics, then they can potentially discover useful patterns and insights, but they face difficulty in examining stream history and performing iterative analysis because data is not persisted or readily re-creatable
Solution Approach 1:
The patent creates a copy of the event stream characteristics by maintaining a profile that captures statistical properties, data types, and behavioral patterns of the stream. This profile serves as a persistent representation that can be examined repeatedly without needing access to the original streaming data, enabling iterative analysis while preserving stream history information.
Solution Approach 2:
The system performs preliminary analysis of the event stream to generate a profile before the developer needs to conduct iterative analysis. This preliminary action captures essential characteristics and makes them available for subsequent repeated examinations, eliminating the need to re-process or re-access the original stream data multiple times.
2Productivity
If CEP developers define business logic without knowledge of stream characteristics, then they can quickly start processing, but they cannot perform effective analysis or tuning because they lack understanding of data sources
Solution Approach 1:
The patent introduces a profile as an intermediary between the event stream and the developer. The profile translates raw stream characteristics into meaningful information about data types, value ranges, and behavioral patterns, making stream characteristics accessible to developers without requiring them to perform complex analysis themselves. This intermediary enables informed query development while maintaining fast processing startup.
3Loss of information
If the system maintains detailed profiles of event streams, then developers gain deeper knowledge about stream behavior, but the system complexity and resource requirements increase
Solution Approach 1:
The patent extracts only the essential characteristics from the event stream to create the profile, rather than maintaining or analyzing all possible stream attributes. By selecting and storing only the most relevant statistical properties and behavioral patterns, the system provides meaningful stream knowledge while keeping the profiling mechanism relatively simple and resource-efficient.
Data Source
AI summary
Certain example embodiments relate to techniques for investigating event streams in complex event processing (CEP) environments. Input events from one or more input event streams and query registration-related events from a registration event stream are received. Query registration-related events are associated with actions taken with respect to queries performed on the input event stream(s). Event-based profiles are developed by subjecting the received input events to a profiling CEP engine. Event-based profiles include data mining related and/or statistical characteristics for each input event stream. Query-based profiles are developed by subjecting the received query registration-related events to the CEP engine. Query-based profiles include data indicative of how relevant the queries performed on the input event stream(s) are and/or how those queries are relevant to the input event stream(s) on which they are performed. Query registration-related events are generated when a query on the input event stream(s) is registered, deregistered, etc.