Telemetry-Based Code Defect Detection Across Validation Rings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud-based systems, it is challenging to determine whether operational failures are due to code defects or noise, especially as new software builds are added, making it difficult to identify and address issues efficiently.
Innovation Solution
A system and method that utilize telemetry data from current and previous software builds to identify operational failure spikes across different development rings, flagging code defects and sending notifications to IT personnel with graphical representations of failure spikes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual analysis of operational failures is performed to determine code defects, then accuracy in identifying code defects is improved, but time consumption and labor costs increase
Solution Approach 1:
The system performs self-service by automatically analyzing operational failure data, comparing telemetry metrics across software builds, and identifying code defects without human intervention. The monitoring service autonomously detects failure spikes, correlates them with specific software builds, and generates defect notifications, eliminating the need for manual analysis while maintaining high accuracy through systematic data comparison and pattern recognition algorithms.
2Measurement precision
If comprehensive telemetry data is collected from all development rings to improve detection accuracy, then measurement precision is improved, but data processing complexity and resource consumption increase
Solution Approach 1:
The system segments the complex telemetry data processing task by dividing development rings into distinct monitoring zones (e.g., staging ring, production ring, canary ring) and processing data from each ring separately. The monitoring service collects telemetry data from multiple rings but analyzes them in a structured sequence, comparing failure patterns across rings to identify genuine code defects while filtering out ring-specific noise. This segmentation reduces processing complexity by organizing data flow and analysis logic into manageable segments.
3Productivity
If frequent software builds are deployed to rapidly add features, then productivity is improved, but the number of operational failures and code defects increases
Solution Approach 1:
The system implements continuous feedback by monitoring operational failures in real-time across multiple development rings and immediately correlating failures with specific software builds. When failure spikes are detected, the system automatically generates notifications to developers, providing rapid feedback about potential code defects. This feedback loop enables quick identification and remediation of issues introduced by frequent deployments, maintaining reliability despite high productivity by ensuring that each new build is quickly validated and problematic builds are promptly addressed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In non-limiting examples of the present disclosure, systems, methods and devices for detecting a code defect associated with a cloud-based service are presented. A correlation between an operation failure spike in an outermost development ring prior to a production environment for a current software build and an operation failure spike in a development ring that precedes the outermost development ring for the current software build may be identified. A determination may be made that an operation failure spike corresponding to the spike in the outermost development ring in a previous software build did not exist. A determination may be made that an operation failure spike corresponding to the development ring that precedes the outermost development ring for a previous software build did not exist. An operation associated with the failure spikes may be flagged as being related to the code defect based on the findings.