Information-Unit Scaling for Ordered Event Stream Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional ordered event stream (OES) data storage systems scale based on the amount of data, which does not account for the varying amounts of information within events of similar data size, leading to inefficient resource utilization and potential overburdening of computing resources, especially when dealing with events containing different levels of information.
Innovation Solution
Implementing information-unit scaling, where the OES system dynamically adjusts its topology based on the computing resources required to access and process information within events, using metrics such as information units (IUs) to determine when to add or remove processing instances and segments, thereby optimizing resource allocation and preserving event ordering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If OES system scales based on data amount, then storage capacity is improved, but computing resource utilization deteriorates due to not accounting for varying information density
Solution Approach 1:
The patent changes the scaling parameter from data volume to information units (IUs), where 1 IU represents a standardized amount of information (e.g., 1 million bits). This allows the system to scale processing instances based on actual information content rather than raw data size, resolving the contradiction between storage capacity and computing resource utilization efficiency
Solution Approach 2:
The patent segments the OES topology into multiple processing instances, each capable of handling a specific number of IUs per second. By dividing the system into discrete units based on information processing capacity rather than data storage capacity, the system can dynamically allocate resources to match actual information processing demands
2Productivity
If OES system increases processing instances to handle high information density events, then information processing capability is improved, but system complexity increases
Solution Approach 1:
The patent implements dynamic scaling of processing instances based on real-time information density measurements. The system can add or remove processing instances as needed to handle varying information loads, maintaining optimal performance without permanently increasing system complexity. This dynamic approach allows the system to adapt to changing conditions rather than being statically over-provisioned
Solution Approach 2:
The patent incorporates feedback mechanisms that monitor information density in the OES and automatically adjust the number of processing instances accordingly. This closed-loop control prevents manual intervention and complex configuration while maintaining optimal processing capability, as the system self-regulates based on actual information content
3Ease of manufacture
If OES system scales based on data volume, then storage expansion is simplified, but event processing performance deteriorates due to overburdening on high information density events
Solution Approach 1:
The patent changes the fundamental scaling parameter from data volume to information units, fundamentally altering how the system expands. Instead of simply adding storage capacity, the system now scales processing instances based on information density, ensuring that event processing performance is maintained even as the system grows. This parameter change addresses both storage expansion and performance simultaneously
Data Source
AI summary
Scaling an ordered event stream (OES) based on an information-unit (IU) metric is disclosed. The IU metric can correspond to an amount of computing resources that can be consumed to access information embodied in event data of an event of the OES. In this regard, the amount of computing resources to access the data of the stream event itself can be distinct from an amount of computing resources employed to access information embodied in the data. As such, where an external application, e.g., a reader, a writer, etc., can connect to an OES data storage system, enabling the OES to be scaled in response to burdening of computing resources accessing event information, rather than merely event data, can aid in preservation of an ordering of events accessed from the OES.