Code Regression Detection via Time-Series ML Classification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud-based systems face challenges in distinguishing between code regressions, network issues, and hardware problems, and determining their urgency, leading to inefficiencies in identifying and addressing service issues.

Innovation Solution

A three-layer approach utilizing unsupervised machine learning, neural networks, and language processing models to classify telemetry data, reducing false positives and focusing human review on critical issues by identifying and categorizing code regressions, network issues, and hardware problems based on severity and urgency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional monitoring methods are used to track cloud-based system operations, then system operations can be monitored, but it is difficult to determine whether operational failures are related to code defects, network issues, or hardware problems

Engineering Contradiction:

Improveissue classification accuracyVSAvoidmonitoring system complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the monitoring system into three distinct layers: (1) an unsupervised machine learning model that processes time series data to identify potential issues, (2) a neural network that classifies the nature of issues (code defects, network issues, hardware problems), and (3) a language processing model that determines severity and urgency. This segmentation allows each layer to specialize in specific classification tasks, improving measurement precision while managing complexity through modular architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces machine learning models and neural networks as intermediaries between raw telemetry data and human reviewers. These intermediaries automatically classify and prioritize issues, transforming unstructured operational data into structured classifications that indicate the type and severity of problems, thereby improving classification accuracy without requiring direct human analysis of raw data.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If engineering teams manually review all operational failures to identify issues, then comprehensive issue identification is possible, but processing costs and time consumption increase significantly

Engineering Contradiction:

Improveissue detection reliabilityVSAvoidissue identification time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by using unsupervised machine learning models to pre-process and filter time series data before human review. The system automatically identifies potential issues, classifies their nature, and determines severity levels in advance, so that human reviewers only need to examine pre-filtered and prioritized candidates, significantly reducing review time while maintaining detection reliability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms where the machine learning models continuously learn from operational data and classification results. The system provides structured feedback to engineering teams about identified issues, and this information feeds back into improving the models' classification accuracy over time, creating a self-improving system that reduces both time and cost of issue identification.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If a three-layer machine learning approach is applied to classify telemetry data, then false positives are reduced and critical issues are prioritized, but processing and computational resources increase

Engineering Contradiction:

Improveissue classification accuracyVSAvoidcomputational resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies partial action by using unsupervised machine learning to process only the most relevant features of time series data in the first layer, then applying neural networks only to identified potential issues in the second layer, and using language processing models only for severity classification in the third layer. This staged approach applies computational resources partially and progressively, reducing overall energy consumption while maintaining high classification accuracy.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11720461B2Automated detection of code regressions from time-series data

Publication Date: 2023.08.08 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

In non-limiting examples of the present disclosure, systems, methods and devices for detecting and classifying service issues associated with a cloud-based service are presented. Operational event data for a plurality of operations associated with the cloud-based application service may be monitored. A statistical-based unsupervised machine learning model may be applied to the operational event data. A subset of the operational event data may be tagged as potentially being associated with a code regression, wherein the subset comprises a time series of operational event data. A neural network may be applied to the time series of operational event data, and the time series of operational event data may be flagged for follow-up if the neural network classifies the time series as relating to a positive code regression category.