Application Log Triage via Machine Learning Vector Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in accurately and efficiently identifying the root causes of application problems and failures, particularly in complex scenarios involving coding errors, network issues, and interactions between applications, making it difficult to prevent or remediate these issues in real time without significant processor and memory demands.
Innovation Solution
A system comprising a client, data processing service, web service, notification service, and data prediction service that uses machine learning to triage issues by classifying application logs, training models, and leveraging these models for quick and accurate problem identification and remediation, facilitating real-time or near real-time processing with minimal computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning is performed in real-time to identify root causes of application problems, then measurement precision and reliability are improved, but processor and memory demands increase significantly
Solution Approach 1:
The patent performs processor-intensive machine learning model training and log pattern analysis in advance (prior to deployment), storing the results in a searchable database. This preliminary action enables real-time queries to return quick, accurate results without requiring significant computational resources during actual problem triage, thus resolving the contradiction between high accuracy and low processor/memory demands.
2Measurement precision
If comprehensive log analysis is performed to classify application problems accurately, then measurement precision is improved, but processing time increases
Solution Approach 1:
The system pre-processes and classifies log data into structured patterns and categories before deployment, storing these classifications in a searchable database. During real-time operation, the system queries this pre-classified data rather than performing comprehensive analysis, achieving both high classification accuracy and fast processing times.
Solution Approach 2:
The patent segments the log analysis process into distinct phases: (1) preliminary offline analysis and classification of log patterns, (2) storage of classified data in structured format, and (3) rapid online querying of pre-classified results. This segmentation allows comprehensive analysis to be performed when time is not critical, while real-time triage benefits from pre-computed classifications.
3Productivity
If real-time problem remediation is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent introduces a data prediction service as an intermediary layer between log collection and problem remediation. This service pre-processes logs, performs machine learning analysis, and stores results in a searchable database, acting as a mediator that simplifies the overall system architecture while enabling real-time remediation capabilities.
Solution Approach 2:
By performing problem classification, root cause identification, and remediation strategy formulation in advance (prior to deployment), the system reduces the complexity of real-time operations. The pre-computed results are stored and readily available for rapid deployment when problems occur, improving productivity without proportionally increasing device complexity.
Data Source
AI summary
Classifying a problem associated with an application may include receiving a log including a record of at least one operation performed by at least one application. The classifying may include generating a vector based on the record. The classifying may include comparing the vector to a plurality of trained vectors each being associated with a separate label. The classifying may include determining a most similar one of the plurality of the trained vectors to the vector based on the comparing. The classifying may include assigning the label associated with the most similar one of the plurality of trained vectors to the vector. The classifying may include selecting a recipient from among a plurality of available recipients based on the label assigned by the processor. The classifying may include causing the log to be further processed by the recipient.