Application Regression Detection via Workload-Normalized Error Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, it is challenging to proactively detect application regression due to the difficulty in distinguishing between increased exception records caused by incompatibility or high workloads, as workload measurement is uncertain, leading to delayed detection of programming or configuration issues in user applications.
Innovation Solution
A score function is defined to calculate error percentages based on telemetry records, normalizing them with workload indicators such as processor load and network throughput, and applying machine learning to establish a baseline, enabling proactive detection of application regression by identifying patterns in error percentages over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error telemetry analysis is performed to detect application regression, then reliability of error detection is improved, but difficulty of detecting and measuring worsens due to inability to distinguish between workload-induced exceptions and regression-induced exceptions
Solution Approach 1:
The patent introduces workload indicators (processor load, network throughput, memory usage) as intermediary metrics that mediate between raw exception counts and regression detection. These indicators serve as a bridge to distinguish whether exceptions are caused by high workload or actual application regression, resolving the ambiguity in error telemetry analysis
Solution Approach 2:
The patent transforms the detection approach by changing from analyzing raw exception counts to analyzing normalized error percentages that account for workload parameters. By introducing multiple parameters (workload indicators, time-based patterns, error type distributions) and analyzing their changes over time, the system can reliably detect regression despite the complexity of distinguishing exception sources
2Loss of time
If proactive detection of application regression is implemented, then loss of time is reduced, but device complexity increases due to need for multiple workload indicators and machine learning components
Solution Approach 1:
The patent implements preliminary action by continuously collecting and analyzing workload indicators in the background before regression becomes critical. The system proactively monitors processor load, network throughput, and memory usage continuously, establishing baseline patterns through machine learning, so that when regression occurs, detection happens immediately without waiting for manual intervention or symptom escalation
Solution Approach 2:
The patent creates a multi-functional detection system that simultaneously performs multiple tasks: collecting workload metrics, analyzing error patterns, detecting regression, and providing diagnostics. The machine learning model serves multiple purposes by learning normal behavior patterns that help distinguish various types of exceptions and predict potential regressions, reducing the need for separate specialized components
3Measurement precision
If error percentage calculation normalizes exception records with workload indicators, then measurement precision is improved, but device complexity increases due to machine learning baseline establishment
Solution Approach 1:
The patent implements feedback by using machine learning to continuously learn from historical workload and error data, establishing dynamic baselines that adapt to changing system conditions. The system feeds collected workload indicators and error patterns back into the learning model, which adjusts its understanding of normal versus abnormal behavior, improving measurement precision over time while automating the normalization process
Solution Approach 2:
The normalization system performs self-service by automatically establishing baselines and adjusting normalization parameters without manual intervention. The machine learning model autonomously learns what constitutes normal error rates under different workload conditions and applies appropriate normalization automatically, reducing the need for manual configuration and maintenance of the complex normalization logic
Data Source
AI summary
Computing systems, devices, and associated methods of detecting application regression in a distributed computing system are disclosed herein. In one embodiment, a method includes receiving data representing telemetry records from one or more hosts of the distributed computing system. At least some of the telemetry records are exception records individually indicating an operation by a user application has failed during execution. The method also includes determining a failure rate of executing the operation by the user application while compensating for a workload of the user application in the distributed computing system. A comparison is performed between the determined failure rate and a threshold. Based on the performed comparison, a regression notification can be generated to indicate that application regression has occurred notwithstanding the workload of the user application in the distributed computing system.