Automated KPI Generation from Software Logs via ML Mapping

Solution Overview

Problem

Conventional methods for generating Key Performance Indicators (KPIs) for software are time-consuming, expensive, and often miss important factors for monitoring software performance and health, relying on unspecific steps and best practices.

Innovation Solution

A method and system using a Machine Learning (ML) model to create a mapping of software log lines to issue IDs, generating a mapping database with hash IDs, and identifying relevant sentences to automatically generate KPIs based on debugging information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional methods are used to generate KPIs, then developers or analysts can create KPIs based on best practices, but the process is time-consuming and expensive

Engineering Contradiction:

ImproveKPI generation speedVSAvoidTime required for KPI creation

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system enables self-service by allowing the software product itself to generate its own KPIs through automated analysis of debugging information and log data. The machine learning model processes existing software data to automatically identify and create relevant KPIs without requiring manual intervention from developers or analysts, transforming the KPI generation process from a human-intensive activity to an autonomous system capability.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical manual process of KPI creation with an automated machine learning-based system. Instead of developers or analysts manually analyzing software performance and creating KPIs based on best practices, the system uses machine learning algorithms to automatically process debugging information, log data, and other software metrics to generate KPIs, thereby eliminating the time-consuming manual mechanics of the traditional approach.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of manufacture

If conventional KPI generation methods are used, then developers can create KPIs based on best practices, but the process is difficult and requires extra efforts

Engineering Contradiction:

ImproveEase of KPI creationVSAvoidComplexity of KPI generation process

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The system enables self-service by allowing the software product itself to generate its own KPIs through automated analysis of debugging information and log data. The machine learning model processes existing software data to automatically identify and create relevant KPIs without requiring manual intervention from developers or analysts, transforming the KPI generation process from a human-intensive activity to an autonomous system capability.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical manual process of KPI creation with an automated machine learning-based system. Instead of developers or analysts manually analyzing software performance and creating KPIs based on best practices, the system uses machine learning algorithms to automatically process debugging information, log data, and other software metrics to generate KPIs, thereby eliminating the time-consuming manual mechanics of the traditional approach.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If conventional KPI generation methods are used, then developers or analysts can create KPIs, but important factors for monitoring software performance may be missed

Engineering Contradiction:

ImproveCompleteness of KPI monitoringVSAvoidKPI generation efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent replaces the mechanical manual process of KPI creation with an automated machine learning-based system. Instead of developers or analysts manually analyzing software performance and creating KPIs based on best practices, the system uses machine learning algorithms to automatically process debugging information, log data, and other software metrics to generate KPIs, thereby eliminating the time-consuming manual mechanics of the traditional approach.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system incorporates feedback mechanisms where the machine learning model continuously analyzes software debugging information, log data, and performance metrics to refine and update KPIs. This feedback loop ensures that the KPIs remain relevant and comprehensive, capturing important factors for monitoring software performance that may have been overlooked in manual KPI creation processes.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11281564B2Method and system for generating key performance indicators (KPIs) for software based on debugging information

Publication Date: 2022.03.22 HCL TECH ITAL SPA

AI summary

A method and system for generating Key Performance Indicators (KPIs) for a software based on debugging information is disclosed. In some embodiments, the method includes creating a mapping of each of a plurality of lines in a log of the software to at least one issue Identifier (ID) from a set of issue IDs. The method further includes generating a mapping database consisting of the logs along with their predicted issued ID based on the aforementioned mapping. The method further includes identifying, for each of the set of issue IDs, a set of mapped sentences based on a set of mapped lines. The method further includes generating, for each of the set of issue IDs, a KPI based on the associated set of mapped sentences and the associated set of mapped lines.