Dynamic Event Data Record Aggregation for Network Capacity Management

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems face challenges in managing large volumes of unpredictable event data records, such as during a denial-of-service attack, leading to data loss and latency due to network capacity limitations, and rely on predetermined aggregation methods that compromise data fidelity.

Innovation Solution

A dynamic system that intelligently initiates data aggregation based on real-time network conditions, balancing data volume by adjusting aggregation levels and using a mixed queue approach to deliver both old and new records, ensuring maximum fidelity during normal loads and scaling during event floods.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If event data records are transmitted without aggregation during event floods, then data fidelity is maintained, but network capacity is exceeded causing data loss and latency

Engineering Contradiction:

Improvedata fidelityVSAvoidnetwork capacity utilization

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system dynamically adjusts the aggregation level based on network conditions. During normal operation, records are transmitted with minimal aggregation to maintain fidelity. During event floods, the system automatically increases aggregation levels to reduce volume while preserving essential information, thus adapting to changing network capacity constraints

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the aggregation parameter dynamically based on network conditions. The aggregation level is adjusted as a variable parameter rather than being fixed, allowing the system to optimize between data fidelity and transmission capacity by modifying the aggregation degree in response to network load

Inventive Principle:

Principle #35Parameter changes

2Productivity

If event data records are aggregated using predetermined schemes, then network capacity constraints are addressed, but data fidelity is compromised

Engineering Contradiction:

Improvenetwork capacity managementVSAvoiddata fidelity

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The aggregation scheme transitions from predetermined to dynamic. The system continuously monitors network conditions and adjusts aggregation parameters in real-time, allowing it to maintain higher fidelity when network capacity permits while still providing volume reduction when constraints arise

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system implements feedback mechanisms where network performance metrics are monitored and used to adjust aggregation levels. This closed-loop control ensures that aggregation intensity is continuously optimized based on actual network conditions, preventing both over-aggregation and unnecessary data transmission

Inventive Principle:

Principle #23Feedback

3Productivity

If excess event data records are purged, then network capacity is preserved, but data loss occurs

Engineering Contradiction:

Improvenetwork capacity preservationVSAvoiddata loss

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system converts the potentially harmful excess data into beneficial aggregated information. Rather than purging records, the aggregation process transforms large volumes of redundant or similar records into condensed summaries that preserve essential patterns and insights while reducing volume

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The system selectively discards redundant information through aggregation while recovering and preserving essential data patterns. The aggregation process identifies and eliminates duplicate or highly similar records while maintaining representative information that captures the essence of the event data

Inventive Principle:

Principle #34Discarding and recovering

4Quantity of substance

If aggregation levels are increased during event floods, then data volume is reduced, but data fidelity decreases

Engineering Contradiction:

Improvedata volume reductionVSAvoiddata fidelity

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The aggregation level is made dynamic rather than fixed. The system adjusts aggregation intensity in real-time based on network capacity availability, allowing it to reduce data volume during floods while maintaining higher fidelity when network conditions improve

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The aggregation parameter is changed dynamically based on network conditions. The system modifies the degree of aggregation as a variable parameter, increasing it during event floods to reduce volume and decreasing it when network capacity allows, thus optimizing the trade-off between volume reduction and fidelity preservation

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10103964B2Managing large volumes of event data records

Publication Date: 2018.10.16 AT&T INTELLECTUAL PROPERTY I L P

AI summary

A network device that operates as an analysis platform for analysis of event data records that can provide a flexible approach to event data record aggregation. For example, aggregation can be flexibly turned on or off and dynamically adjusted based on event record volume and other factors such as network capacity or throughput. Devices that are instructed to aggregate records can also be instructed to archive the raw records, e.g., to maintain a full fidelity log of events. Devices can further be instructed to utilize a mixed queue approach to determine an order to deliver those records that includes both older records and newer records.