Dynamic Rule Engine State Cache for Consistent Event Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in ensuring consistent results when applying rules in testing and production environments, requiring static testing environments and potentially altering developer workflows to accommodate complex event processing.
Innovation Solution
A method and system that allows for dynamic access to data sources during rule development and testing, enabling consistent rule application across single unit, batch, and production modes without the need for static testing environments, by using a state variable cache and iterative processing of data units to ensure accurate state updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a static testing environment is used to apply rules, then testing consistency can be maintained, but the system cannot dynamically access data sources during rule development and testing
Solution Approach 1:
The system transitions from a static testing environment to a dynamic one by enabling real-time access to data sources during rule development and testing. The rule application process can now dynamically query external data sources while maintaining consistent results through proper state variable management and ordering.
Solution Approach 2:
The patent introduces an intermediary layer (the processing module with state variable cache) that mediates between dynamic data source access and consistent rule application results. This intermediary manages the complexity of dynamic access while presenting a consistent interface to the rule engine.
2Productivity
If complex event processing is implemented, then rule processing capability is improved, but developer workflows must be altered to accommodate the complexity
Solution Approach 1:
The system provides self-service capabilities by automatically managing state variables, caching mechanisms, and data source access. Developers can define rules using existing workflows without needing to manually manage the underlying complex event processing state, as the system handles these details automatically.
Solution Approach 2:
The processing module acts as an intermediary that shields developers from the complexity of complex event processing. It manages state variables and data source access automatically, allowing developers to focus on rule logic without altering their workflows to accommodate CEP complexity.
3Productivity
If state variables are cached for performance improvement, then processing speed increases, but memory resources are consumed
Solution Approach 1:
The system dynamically adjusts caching parameters such as cache size, expiration times, and eviction policies based on available memory resources and processing requirements. This allows the system to optimize the balance between processing speed and memory consumption by changing caching behavior adaptively.
Solution Approach 2:
The state variable cache implements discard and recover mechanisms where less recently used or stale state variables are automatically evicted from memory to make room for new data. This allows the system to maintain high processing speed while managing memory resources efficiently through selective discarding and subsequent recovery when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for applying a rule to data includes receiving data from a data source and processing a particular data unit of an ordered number of data units derived at least in part from the received data. The processing includes determining a set of data units including an ordered subset of data units from the ordered number of data units, each data unit of the ordered subset being related to the particular data unit and occurring before the particular data unit in the ordered number of data units. Prior to applying the rule the state variable is updated to a state that would result from iterating in order through the set of data units and, for each data unit of the ordered subset, applying the rule to the data unit. The rule is then applied to the particular data unit including reading the updated value of the state variable.