Hybrid ML Issue Recommendation for Cloud Log Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern cloud-based systems with heterogeneous architectures generate complex and voluminous log data, making it challenging to prioritize issues and errors effectively, leading to inefficiencies and resource misallocation.
Innovation Solution
A system and method for generating machine learning recommendations using log data, which processes event streams to extract features and apply hybrid collaborative and content-based filtering algorithms to identify impactful issues across different system architectures, enabling proactive issue prioritization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning algorithms process voluminous log data from heterogeneous cloud systems, then issue prioritization accuracy is improved, but data processing complexity and computational resource requirements increase
Solution Approach 1:
The patent segments the data processing pipeline into distinct stages: log data ingestion, feature extraction, collaborative filtering, content-based filtering, and recommendation generation. Each stage processes specific portions of the data independently, reducing overall complexity while maintaining accuracy through staged processing of the voluminous log data from heterogeneous cloud systems.
Solution Approach 2:
The patent introduces feature extraction as an intermediary layer between raw log data and machine learning algorithms. This intermediary transforms complex heterogeneous log data into structured features that can be efficiently processed by ML models, reducing computational complexity while preserving the essential information needed for accurate issue prioritization.
2Measurement precision
If hybrid collaborative and content-based filtering is used to generate issue recommendations, then recommendation accuracy is improved, but algorithm complexity and processing time increase
Solution Approach 1:
The patent merges collaborative filtering and content-based filtering into a hybrid recommendation system. The collaborative filtering component analyzes user behavior patterns across heterogeneous cloud systems, while the content-based filtering component processes feature extracts from log data. By combining both approaches, the system achieves higher recommendation accuracy through complementary strengths of both filtering methods.
Solution Approach 2:
The patent performs preliminary feature extraction and preprocessing of log data before feeding it into the hybrid filtering algorithms. This preliminary action reduces the computational burden on the ML models by preparing the data in advance, thereby reducing processing time while maintaining high recommendation accuracy through the hybrid approach.
3Speed
If real-time processing of event streams is implemented, then issue detection speed is improved, but system resource consumption increases
Solution Approach 1:
The patent implements continuous processing of event streams from cloud systems using streaming architectures. The system maintains continuous ingestion and processing of log data without interruption, enabling real-time issue detection. This continuous action allows the system to detect issues as they occur while managing resource consumption through efficient streaming processing techniques that avoid batch processing overhead.
Data Source
AI summary
Embodiments generate machine learning recommendations using log data. Log data can be ingested to generate an event stream for cloud systems, where each of the cloud systems comprises a combination of components, and the cloud systems present heterogenous system architectures. The generated event streams can be processed to generate a data set, where the data set include issue labels for issues experienced by the cloud systems. Features from the generated data set can be extracted. Issue recommendations can be generated using machine learning algorithms based on the extracted features and the generated data set, where the issue recommendations are generated using a hybrid of collaborative based machine learning filtering and content based machine learning filtering.