NLP Metric Relevance Scoring for Faster Root-Cause Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud providers face challenges in identifying a smaller set of relevant metrics for root-cause analysis due to varying customer-defined names for the same or similar metrics across different workspaces, leading to resource-intensive and time-consuming analysis processes.
Innovation Solution
A system that determines relevance scores by converting customer-defined metric names into mathematical representations using natural language processing, allowing for efficient filtering and ingestion of a smaller set of relevant metrics through training and inference stages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all available metrics are queried and ingested for root-cause analysis, then the analysis is comprehensive, but the analysis time becomes unreasonably long (over thirty minutes to an hour)
Solution Approach 1:
The system extracts only the most relevant metrics from the large set of available metrics by comparing metric names using natural language processing. This extraction process identifies a smaller subset of metrics that are most likely to be relevant to the incident, thereby reducing analysis time while maintaining sufficient comprehensiveness for effective root-cause analysis.
Solution Approach 2:
The system changes the parameter of metric selection from including all metrics to including only those with high relevance scores. By transforming metric names into numerical representations and comparing them to incident descriptions, the system dynamically determines which metrics to include, balancing comprehensiveness with speed.
2Loss of time
If a smaller set of metrics is selected for faster analysis, then the analysis time is reduced, but the difficulty of identifying which metrics are relevant increases due to varying customer-defined names
Solution Approach 1:
The system introduces an intermediary process that transforms metric names into numerical representations using natural language processing. This intermediary transformation enables automated comparison and relevance scoring, overcoming the difficulty of varying customer-defined names and allowing systematic identification of relevant metrics without manual intervention.
Solution Approach 2:
The system replaces manual identification of relevant metrics with an automated computational process. By substituting human judgment with algorithmic comparison of numerical representations, the system efficiently identifies relevant metrics despite name variations across different customers' workspaces.
3Measurement precision
If manual methods are used to identify relevant metrics for each workspace, then the analysis can be tailored to specific incidents, but the process becomes resource-intensive and difficult to scale
Solution Approach 1:
The system creates a universal solution that works across all customer workspaces by using natural language processing to compare metric names to incident descriptions. This universal approach maintains accuracy in identifying relevant metrics while enabling scalable deployment across thousands of workspaces without requiring manual configuration for each incident.
Solution Approach 2:
The system enables self-service automated metric selection that does not require manual intervention for each incident or workspace. The automated process independently identifies relevant metrics by comparing numerical representations, allowing the system to serve itself and scale efficiently across multiple workspaces while maintaining consistent accuracy.
Data Source
AI summary
Disclosed herein is a system for determining scores that are usable to filter a larger set of metrics (e.g., thousands of metrics) down to a smaller set of relevant metrics (e.g., hundreds of metrics) that can be more efficiently queried and ingested for root-cause analysis of an incident. During a training stage, the system analyzes known incidents and converts the names of the metrics, as described via customer-defined words, into mathematical representations (e.g., word embedding featurization vectors). When a new metric with a new name is received for a new incident, the system implements an incident inference stage during which the new name is converted into a new mathematical representation. The system compares the new mathematical representation to the mathematical representations to identify a similar mathematical representation. The system retrieves the score for the metric associated with the similar mathematical representation and assigns the retrieved score to the new metric.