Anomaly Detection via Event Probability Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining anomalous events in data centers inaccurately identify events as anomalous based on field values with the lowest probability of occurrence, failing to account for the overall probability of event occurrence, which can lead to misclassification.
Innovation Solution
Determine the probability of occurrence for each field value and then for each event, using these probabilities to identify anomalous events, providing a more accurate method for anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing methods use field value probability to determine anomalous events, then the process is simple, but the accuracy of anomaly detection is poor
Solution Approach 1:
The patent segments the probability calculation into two distinct levels: field value probability (P(FV)) and event probability (P(Event)). This segmentation allows the system to first calculate simple field value probabilities, then combine them through a structured process to derive overall event probabilities, resolving the contradiction by making the complex calculation manageable through systematic division of the computation process
Solution Approach 2:
The patent changes the parameter from field value probability to event probability. Instead of relying on the probability of individual field values, the system calculates the probability of the entire event occurring by combining field value probabilities. This parameter change directly improves anomaly detection accuracy while maintaining a systematic approach to the calculation complexity
2Loss of information
If more data is preserved in unstructured form, then data availability is improved, but indexing and searching operations become difficult
Solution Approach 1:
The patent adds a probability dimension to the unstructured data. By calculating and associating probability values with events and field values, the system creates a new dimensional attribute that enables sorting, filtering, and prioritization of data without requiring traditional indexing structures. This dimensional addition allows efficient retrieval and analysis while preserving data in unstructured form
Data Source
AI summary
Methods and systems for determining event probabilities and anomalous events are provided. In one implementation, a method includes: receiving source data, where the source data is configured as a plurality of events with associated timestamps; searching the source data, where the searching provides a search result including N events from the plurality of events, where N is an integer greater than one, where each event of the N events includes a plurality of field values, where at least one event of the N events can include one or more categorical field values and one or more numerical field values; and for an event of the N events, determining a probability of occurrence for each field value of the plurality of field values; and using probabilities determined for the plurality of field values, determining a probability of occurrence for the event.