Real-Time Anomaly Streaming Module for Live Data Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing computing systems face challenges in identifying and resolving incidents promptly due to latency issues, scalability problems, and delayed anomaly detection during software updates, leading to increased costs and service disruptions.

Innovation Solution

A real-time anomaly streaming module that utilizes a unified stream-processing and batch-processing framework with tumbling and sliding windows, matrix profiling, and advanced anomaly detection algorithms to identify anomalies in live streaming data, enabling immediate detection and response.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If traditional batch processing is used for data analysis, then system complexity is reduced, but detection speed and real-time response capability deteriorate

Engineering Contradiction:

Improvedetection speedVSAvoidsystem complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The patent segments the data processing system into multiple streaming modules that operate in parallel, each handling specific data streams independently. This segmentation enables real-time processing without requiring a complete system redesign, thus improving detection speed while managing complexity through modular architecture

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts processing parameters and resource allocation based on incoming data characteristics and anomaly detection needs. This dynamic adaptation allows the system to optimize detection speed in real-time while maintaining manageable complexity through flexible resource management

Inventive Principle:

Principle #15Dynamics

2Loss of time

If real-time streaming processing is implemented, then anomaly detection timeliness is improved, but processing resource consumption increases

Engineering Contradiction:

Improveanomaly detection timelinessVSAvoidprocessing resource consumption

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The system applies partial processing to normal data streams while allocating excessive processing resources only when anomalies are detected or suspected. This selective resource allocation maintains timely anomaly detection while reducing overall processing resource consumption by avoiding full-intensity processing of all data continuously

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The streaming processing system implements self-service mechanisms where modules automatically adjust their resource consumption based on data patterns and anomaly likelihood. This self-regulation reduces processing resource consumption while maintaining detection timeliness without requiring constant external resource management

Inventive Principle:

Principle #25Self-service

3Measurement precision

If comprehensive data analysis is performed, then anomaly detection accuracy is improved, but processing latency increases

Engineering Contradiction:

Improveanomaly detection accuracyVSAvoidprocessing latency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The analysis is segmented into multiple streaming modules that process different aspects of data in parallel. Each module performs specialized analysis on its assigned data stream, enabling comprehensive anomaly detection accuracy while reducing overall processing latency through concurrent execution rather than sequential processing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary filtering and preprocessing of data streams before detailed anomaly analysis. This preliminary action prepares data in advance for faster processing, improving detection accuracy by ensuring data is ready for analysis while reducing latency by avoiding last-minute processing delays

Inventive Principle:

Principle #10Preliminary action

4Productivity

If manual incident investigation is used, then analysis depth is improved, but resolution time increases

Engineering Contradiction:

Improveincident resolution speedVSAvoidanalysis depth

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system implements feedback loops where anomaly detection results automatically trigger targeted investigations and where investigation findings feed back to refine detection algorithms. This feedback mechanism maintains analysis depth by ensuring thorough examination of detected anomalies while improving resolution speed by avoiding manual review of all data and focusing resources on confirmed issues

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20250110820A1Systems and methods for a real time anomaly streaming module

Publication Date: 2025.04.03 FIDELITY INFORMATION SERVICES LLC

AI summary

A method for processing live streaming data includes assigning one or more data to one or more windows based on group level characteristics of the one or more data, assigning the one or more data to one or more sub-windows based on one or more time stamps, the one or more sub-windows inside the windows, creating a data count for each of the sub-windows, creating a first time series from the data count of each of the sub-windows, comparing the time series to a second time series of data, wherein the second time series of data is previous time series data, identifying presence of one or more anomalies based on the comparison of first time series to the second time series, alerting a user to the presence of anomalies based on a result of the comparison, and modifying the second time series based on the first time series.