Health Score Computation Using Engineered Log Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional human-assisted machine learning solutions for monitoring IT systems face challenges due to the use of raw data with sparse feature values and data quality issues, leading to complex models that are difficult to maintain and track, and often fail to capture holistic system behavior effectively.
Innovation Solution
The implementation of additional engineered features over time-series counters, such as entropy, unique values, and clustered counts, which are processed using machine learning models to compute a health score for the monitored system, reducing model complexity and improving data quality and monitoring capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional ML solutions use raw data with sparse feature values, then the monitoring solution can be implemented, but the ML models become unnecessarily complicated and difficult to maintain
Solution Approach 1:
The patent applies preliminary action by computing additional features (entropy, clustered counts, unique values, modeled operations) from raw log data before feeding it to the ML model. This preprocessing step transforms sparse, raw data into enriched features that capture system behavior patterns, allowing the ML model to achieve high accuracy without becoming unnecessarily complex.
Solution Approach 2:
The patent introduces additional engineered features as intermediaries between raw log data and the ML model. These features (entropy function, clustered counts, unique values, modeled operations) act as mediators that bridge the gap between raw data and model input, improving data quality and enabling simpler, more maintainable models while preserving monitoring accuracy.
2Reliability
If traditional ML solutions use raw data with data quality issues, then the monitoring solution can be implemented, but the models become difficult to track and maintain
Solution Approach 1:
The patent applies preliminary action by computing additional features (entropy, clustered counts, unique values, modeled operations) from raw log data before feeding it to the ML model. This preprocessing step transforms sparse, raw data into enriched features that capture system behavior patterns, allowing the ML model to achieve high accuracy without becoming unnecessarily complex.
Solution Approach 2:
The patent introduces additional engineered features as intermediaries between raw log data and the ML model. These features (entropy function, clustered counts, unique values, modeled operations) act as mediators that bridge the gap between raw data and model input, improving data quality and enabling simpler, more maintainable models while preserving monitoring accuracy.
3Device complexity
If traditional ML solutions take advantage of data only in its raw form, then the implementation is simple, but the solutions fail to capture holistic system behavior effectively
Solution Approach 1:
The patent applies parameter changes by transforming raw log data into multiple derived features with different characteristics. The entropy function captures distribution patterns, clustered counts capture entity groupings, unique values capture diversity, and modeled operations capture correlations. This multi-parameter transformation enriches the data representation, enabling the ML model to capture holistic system behavior while maintaining processing efficiency.
Data Source
AI summary
Techniques are provided for system operational analytics using additional features over time-series counters for health score computation. An exemplary method comprises: obtaining log data from data sources of a monitored system; applying a counting function to the log data to obtain time-series counters for a plurality of distinct features within the log data; applying an additional function to the time-series counters for the plurality of distinct features; and processing an output of the additional function using a machine learning model to obtain a health score for the monitored system based on the output of the additional function. The additional function comprises, for example, an entropy function representing a load balancing of a plurality of devices in the monitored system; one or more clustered counts for a plurality of entities in the monitored system; a number of unique values; and/or one or more modeled operations based on correlations between a plurality of different operations in the monitored system.