Cross-Entropy Ranking for Anomaly Source Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying the cause of anomalies in time series data from conversational metrics is inefficient due to the need to review numerous conversations, leading to high computational complexity and time consumption.
Innovation Solution
A system and method using cross-entropy ranking of textual items, where a language model is trained on relevant textual data from a time window associated with the anomaly, and textual items are ranked based on their cross-entropy values to quickly identify the most likely conversations that triggered the anomaly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all conversations at the time of anomaly are reviewed to determine the cause, then the completeness of anomaly analysis is improved, but the time consumption and computational complexity increase significantly
Solution Approach 1:
The patent extracts only the most relevant conversations from the entire dataset by using cross-entropy ranking to identify and select top-k conversations that are most likely to have triggered the anomaly. This extraction approach maintains analysis completeness while significantly reducing the volume of data that needs manual review.
Solution Approach 2:
The patent introduces cross-entropy ranking as an intermediary mechanism between the anomaly detection system and manual analysis. This intermediary automatically scores and ranks conversations based on their linguistic surprisal, serving as a filter that prioritizes the most relevant conversations for human review.
2Measurement precision
If all conversations at the time of anomaly are reviewed to determine the cause, then the completeness of anomaly analysis is improved, but the computational complexity increases significantly
Solution Approach 1:
The patent extracts only the most relevant conversations from the entire dataset by using cross-entropy ranking to identify and select top-k conversations that are most likely to have triggered the anomaly. This extraction approach maintains analysis completeness while significantly reducing the volume of data that needs manual review.
Solution Approach 2:
The patent replaces manual mechanical review of conversations with an automated computational system that uses cross-entropy calculation to objectively rank conversations. This substitution reduces human computational effort while maintaining or improving analysis thoroughness through systematic scoring.
3Productivity
If cross-entropy ranking is used to identify top-k conversations, then the processing time is reduced, but the risk of missing relevant conversations increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the system allows manual review and verification of the top-k ranked conversations. Analysts can provide feedback on whether the ranked conversations accurately reflect the anomaly cause, and this feedback can be used to refine future ranking calculations, ensuring both speed and accuracy.
Solution Approach 2:
The patent applies partial action by analyzing only the top-k most surprising conversations rather than all conversations. This partial approach is justified by the cross-entropy ranking that prioritizes the most likely anomaly-triggering conversations, achieving sufficient accuracy without requiring complete review of all data.
Data Source
AI summary
A framework for reducing the number of textual items reviewed to determine the source of or reason for an anomaly in a time series that is used to track metrics in textual data is provided. According the framework, textual items in a time window corresponding to the anomaly are ranked according to the cross-entropy as determined by applying a language model to the relevant textual items and ranking textual items that most likely triggered an anomaly in time series data based on the cross-entropy value. In an aspect, a predetermined number of textual items having the highest cross-entropy are provided or all textual items having cross-entropy value higher than predetermine threshold are provided.