Software Deployment Fault Detection Using Weak Supervision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting faulty software deployments are time-consuming and prone to false positives/negatives due to the manual and simplistic nature of performance monitoring telemetry.
Innovation Solution
A system utilizing a sequence of machine learning models with weak supervision to generate non-binary classifications, which are then converted into strong labels for training, enabling rapid detection of defects in software deployments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual comparison of telemetry to baseline is used, then detection accuracy can be maintained, but the process is time-consuming and error-prone
Solution Approach 1:
The patent replaces manual mechanical comparison processes with automated machine learning models. The system uses ML models to automatically analyze telemetry data and detect deployment faults, eliminating the need for manual comparison while maintaining or improving detection accuracy through intelligent pattern recognition and anomaly detection algorithms.
Solution Approach 2:
The system enables self-service through automated fault detection where the machine learning models continuously monitor telemetry data without human intervention. The models automatically learn from historical data, adapt to changing patterns, and independently perform fault detection, reducing both time consumption and manual effort while maintaining high detection accuracy.
2Ease of manufacture
If simple monitors are used for automation, then implementation is easier, but false positives and false negatives increase
Solution Approach 1:
The patent changes the parameters of detection by transitioning from simple threshold-based monitors to complex machine learning models that consider multiple features, temporal patterns, and contextual information. This parameter transformation enables the system to maintain implementation simplicity while significantly improving detection reliability by reducing false positives and false negatives through sophisticated pattern recognition.
Solution Approach 2:
The system employs composite detection approaches by combining multiple machine learning models, feature sets, and analysis methods into a unified fault detection system. This composite approach integrates the strengths of different detection techniques while mitigating their individual weaknesses, thereby improving overall reliability without complicating the implementation process.
3Measurement precision
If multiple models are executed in sequence, then detection precision improves, but system complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the fault detection task into multiple specialized models, each responsible for detecting specific types of faults or analyzing particular aspects of telemetry data. This segmentation allows the system to achieve high detection precision through specialized analysis while managing complexity by organizing models in a structured sequence with clear roles and responsibilities.
Solution Approach 2:
The system implements dynamics by making the model execution sequence adaptive rather than fixed. The system dynamically selects and executes models based on the current deployment context, telemetry data characteristics, and detected anomalies. This dynamic approach enables high detection precision by applying the most relevant models to each situation while managing complexity through intelligent decision-making rather than rigid predetermined sequences.
Data Source
AI summary
The technology disclosed herein provides a framework for quickly detecting faulty software deployments, using a sequence of different analysis models executed at different time increments after the deployment. The analyses may include different machine learning models. Periodically, the system collects data on each deployment during the given period, and applies a set of labelling functions to generate non-binary classifications. The non-binary classifications are used to generate labels using weak supervision, and the labels are used for training a supervised machine learning model. The trained models may be used in the sequence of different analyses executed for future software deployments.