Abnormality Detection Using Probability-Based Dissimilarity

Solution Overview

Problem

Conventional abnormality detection techniques fail to accurately distinguish between normal and abnormal data when features include defects or outliers, and they are unstable in high-dimensional data environments due to the influence of components with large distances.

Innovation Solution

An abnormality detection system that generates a generation distribution of features, creates reference data, calculates a dissimilarity degree based on the probability of features being closer to the reference data, and determines abnormality based on this probability, thereby suppressing the impact of defects or outliers and maintaining stability in high-dimensional data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If conventional distance-based methods are used for abnormality detection, then the detection process is simple, but the determination becomes unstable when features include defects or outliers

Engineering Contradiction:

Improvedetection process simplicityVSAvoiddetermination stability

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent changes the parameter for measuring dissimilarity from conventional distance metrics to a probability-based dissimilarity degree. Instead of using Euclidean distance or other standard distance measures, the system calculates the probability that features are nearer to reference data than to each other, fundamentally changing how similarity is quantified to reduce sensitivity to outliers

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent introduces a probability calculation as an intermediary between feature comparison and abnormality determination. Rather than directly comparing distances, the system uses probability as a mediator to assess whether features are nearer to reference data, providing a more robust intermediate measure that accounts for distribution characteristics

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of manufacture

If conventional distance metrics are used for high-dimensional data, then the calculation is straightforward, but the determination of similarity becomes unstable

Engineering Contradiction:

Improvecalculation simplicityVSAvoidsimilarity determination stability

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The patent changes the parameter for measuring dissimilarity from conventional distance metrics to a probability-based dissimilarity degree. Instead of using Euclidean distance or other standard distance measures, the system calculates the probability that features are nearer to reference data than to each other, fundamentally changing how similarity is quantified to reduce sensitivity to outliers

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent transitions from measuring similarity in the original feature space to measuring it in a probability space. By converting distance-based similarity into probability-based dissimilarity, the system effectively operates in a transformed dimensional space where the relationships between data points are more stable and meaningful

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Adaptability or versatility

If distance-based abnormality detection is applied, then the method is universally applicable, but defects or outliers in features cause false abnormality determination

Engineering Contradiction:

Improvemethod applicabilityVSAvoidabnormality determination accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent changes the parameter for measuring dissimilarity from conventional distance metrics to a probability-based dissimilarity degree. Instead of using Euclidean distance or other standard distance measures, the system calculates the probability that features are nearer to reference data than to each other, fundamentally changing how similarity is quantified to reduce sensitivity to outliers

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent converts the harmful effect of outliers and defects into a benefit by using probability-based dissimilarity. The probability calculation naturally handles outliers by considering the overall distribution rather than being dominated by extreme values, turning what was previously a source of error into a feature that enhances robustness

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Data Source

PatentUS8584000B2Abnormality detection system, abnormality detection method, and abnormality detection program storage medium

Publication Date: 2013.11.12 NEC CORP

AI summary

Even if data includes a defect or an outlier in features thereof, the influence of the defect or the outlier of the features is suppressed to perform a highly precise abnormality detection, and data including high-dimensional features is processable to accomplish the highly stable detection of an abnormality. The abnormality detection system which detects abnormal data in a data sequence including data of multi-dimensional features, and the system includes storing or generating a generation distribution of features of the data and reference data indicative of normal data; obtaining, every piece of the data sequence, a probability that when features are virtually generated from the generation distribution, the features are nearer to the reference data than the features of each piece of the data; and taking the probability as a one-dimensional dissimilarity degree between each piece of the data and the reference data, thereby determining abnormal data.