User-Defined Anomaly Detection Rules in Streaming Event Queues

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing anomaly detection technologies in data platforms suffer from delays in detecting anomalies due to batch processing, and lack user-friendly interfaces for configuring and managing anomaly detection rules, making them cumbersome for non-technical users.

Innovation Solution

The software technology enables the creation and execution of user-defined anomaly detection rules through a user-friendly interface, allowing for real-time anomaly detection by iteratively applying rules to streaming events and generating targeted notifications to users.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If batch processing is used for anomaly detection, then system complexity is reduced, but detection speed and responsiveness deteriorate

Engineering Contradiction:

Improvesystem complexityVSAvoiddetection speed

Core Design Contradiction:

Device complexityVSSpeed

Solution Approach 1:

The system dynamically switches between batch processing and streaming processing modes. The anomaly detection rules are applied iteratively to streaming event queues in near real-time, allowing the system to adapt to changing data flow conditions while maintaining operational simplicity through automated rule management.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The anomaly detection system is segmented into independent rule components that can be individually created, stored, and executed. Each anomaly detection rule is stored as a separate data representation in a data store, allowing parallel processing and reducing overall system complexity while enabling faster detection through concurrent rule evaluation.

Inventive Principle:

Principle #1Segmentation

2Ease of operation

If user-friendly interface is implemented for configuring anomaly detection rules, then ease of operation is improved, but device complexity increases

Engineering Contradiction:

Improveease of operationVSAvoiddevice complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system provides self-service capabilities through automated rule creation and management. Users can interact with a user interface to define anomaly conditions, and the system automatically generates the corresponding streaming query statements and applies them to the streaming event queues, eliminating the need for manual configuration of complex processing logic.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The user interface acts as an intermediary layer between the user and the complex streaming query processing system. It translates user-friendly rule definitions into the technical streaming query statements required by the underlying data processing engine, shielding users from complexity while enabling powerful anomaly detection capabilities.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Measurement precision

If streaming query statements are applied iteratively to streaming event queues, then detection precision is improved, but processing time increases

Engineering Contradiction:

Improvedetection precisionVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The anomaly detection rules are applied continuously to the streaming event queues rather than in discrete batches. The iterative application of streaming query statements maintains a continuous detection process that evaluates anomalies as events flow through the system, ensuring timely detection without sacrificing precision through repeated evaluation.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system employs periodic evaluation of anomaly conditions at defined intervals within the streaming process. This allows the iterative application of detection rules to occur at optimized frequencies, balancing detection precision with processing efficiency by evaluating anomalies at critical decision points rather than continuously without interruption.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS12326776B2Computing systems and methods for creating and executing user-defined anomaly detection rules and generating notifications for detected anomalies

Publication Date: 2025.06.10 CAPITAL ONE FINANCIAL CORP

AI summary

A computing platform may be installed with software technology for creating and executing user-defined anomaly detection rules that configures the computing platform to: (1) receive, from a client device, data defining a given anomaly detection rule that has been created by a user, wherein the given anomaly detection rule comprises at least one anomaly condition that is to be applied to at least one streaming event queue, (2) store a data representation of the given anomaly detection rule in a data store, (3) convert the data representation of the given anomaly detection rule to a streaming query statement, (4) iteratively apply the streaming query statement to the at least one streaming event queue, and (5) while iteratively applying the streaming query statement, make at least one determination that the at least one anomaly condition is satisfied and then cause at least one anomaly notification to be issued to the user.