Anomaly Detection Apparatus Using Scale Transformation for Concept Drift

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Anomaly detection in IT systems is hindered by sudden concept drifts caused by events like configuration changes, log file deletions, or transfers, leading to delayed detection or false alarms due to the need for relearning models with new data.

Innovation Solution

An anomaly detection apparatus that applies a scale transformation to correct predicted data after an event, comparing it with measured data to promptly detect anomalies by selecting between scale and shift transformations based on error minimization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If relearning is performed using measured data after event detection, then model accuracy is improved, but anomaly detection promptness deteriorates due to time consumption

Engineering Contradiction:

Improveanomaly detection accuracyVSAvoidanomaly detection delay

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-calculating and storing transformation parameters (scale and shift values) before events occur. When an event is detected, the system immediately applies these pre-prepared parameters to correct predicted values without needing to perform time-consuming relearning, thus resolving the contradiction between maintaining accuracy and ensuring prompt detection

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes parameters by transforming predicted values using scale and shift corrections instead of retraining the entire model. This parameter-based correction approach maintains detection accuracy while dramatically reducing the time required to adapt to concept drift, directly addressing the time-accuracy tradeoff

Inventive Principle:

Principle #35Parameter changes

2Productivity

If no correction is applied to predicted values, then anomaly detection is performed quickly, but false detections increase due to concept drift

Engineering Contradiction:

Improvedetection speedVSAvoiddetection accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces an intermediary correction mechanism that acts between the original predicted values and the final anomaly detection results. By applying scale and shift corrections as an intermediate step, the system maintains both speed (by avoiding full relearning) and reliability (by correcting concept drift effects), thus resolving the contradiction between detection speed and accuracy

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11829226B2Anomaly detection apparatus, anomaly detection method, and anomaly detection program

Publication Date: 2023.11.28 HITACHI LTD

AI summary

To improve promptness of anomaly detection after occurrence of an event, provided is an anomaly detection apparatus including a processor that executes a program and a storage device that stores the program. The processor executes a correction process of applying a scale transformation to correct second predicted data in time-series first predicted data of a monitoring target, the second predicted data including data after occurrence time of a specific event, and a detection process of detecting an anomaly of the monitoring target based on the second predicted data corrected in the correction process and based on second measured data in time-series first measured data of the monitoring target, the second measured data including data after the occurrence time of the specific event.