Tuple-Based Event Processing for Memory-Constrained CEP
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing complex event processing (CEP) techniques face limitations in handling long time ranges due to memory constraints and separation of data and programs, resulting in low real-time performance, while batch processing offers high real-time performance but at the cost of limited time range handling.
Innovation Solution
A data processing apparatus that records events and programs as paired sets, or tuples, allowing for flexible processing without relying on the order of event generation or time intervals, reducing memory requirements and enabling longer time range handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional CEP processes events in real-time with high speed, then response performance is improved, but the time range that can be handled is limited to very narrow duration
Solution Approach 1:
The patent segments the event processing into two distinct phases: a recording phase where events are stored with their timestamps and data, and a processing phase where conditions are evaluated against the recorded events. This segmentation allows the system to handle long time ranges by recording events first and then processing them in a controlled manner, resolving the contradiction between real-time speed and time range handling.
Solution Approach 2:
The patent implements preliminary action by recording events and their timestamps before processing. The system pre-stores event data in a buffer with time information, allowing subsequent processing to evaluate conditions across extended time ranges without requiring real-time processing during the entire duration. This enables long time range handling while maintaining processing efficiency.
2Duration of action of moving object
If batch processing handles long time ranges with high memory capacity, then time range handling is improved, but real-time performance becomes very low
Solution Approach 1:
The patent introduces dynamic processing by allowing the evaluation of conditions to be triggered at different times. The system can process recorded events in real-time when needed, or batch-process them when time range handling is required. This dynamic approach enables the system to adapt between real-time performance and time range handling based on operational requirements, resolving the contradiction between speed and duration.
3Speed
If CEP stores all events in memory for high-speed processing, then processing speed is improved, but memory requirements increase significantly
Solution Approach 1:
The patent extracts only the necessary event information (timestamp and data) into a buffer during recording, rather than storing all event details in memory for processing. The condition evaluation is performed separately using this extracted minimal data structure, significantly reducing memory requirements while maintaining processing speed. This extraction approach resolves the contradiction between processing speed and memory capacity.
Data Source
AI summary
There is provided a data processing apparatus including: a set data recording unit configured to record set data obtained by combining an event representing data which is generated by processing data by a program, with a program which is started when a plurality of events including the event are complete and processes the plurality of events; and a data processing execution unit configured to execute a plurality of programs by a plurality of pieces of set data recorded by the set data recording unit. There is provided a data processing method including: recording set data obtained by combining an event representing data which is generated by processing data by a program, with a program which is started when a plurality of events including the event are complete and processes the plurality of events; and executing a plurality of programs by a plurality of pieces of set data recorded.