Anomalous Metadata Sub-Sequence Detection Using Dynamic Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional models are unable to detect anomalous sub-sequences in system logs and identify root causes of anomalies, and they lack the ability to dynamically adjust their input context window, making it difficult to identify potential issues that may lead to future hardware or software failures.
Innovation Solution
A framework utilizing machine learning models, particularly large language models, is employed to detect anomalous sub-sequences in metadata by leveraging contextual and syntactic learning, and dynamically adjusting the input context length to identify root causes of anomalies in system logs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional models are used to detect anomalies in system logs, then the system is simple and easy to operate, but the accuracy and precision of anomaly detection is insufficient
Solution Approach 1:
The patent introduces an intermediary layer between data collection and anomaly detection. A metadata extraction system processes raw system logs into structured metadata representations, which then feed into the anomaly detection model. This intermediary metadata layer simplifies the detection process by transforming complex unstructured logs into manageable structured data formats that the model can process effectively.
Solution Approach 2:
The patent replaces conventional statistical or rule-based anomaly detection mechanisms with machine learning models, specifically deep learning architectures like transformers and autoencoders. This substitution enables the system to learn complex patterns and anomalies from data automatically, significantly improving detection accuracy while handling the complexity through automated learning rather than manual rule configuration.
2Adaptability or versatility
If the input context window is fixed in conventional models, then the model structure is simple, but the ability to identify root causes of anomalies is limited
Solution Approach 1:
The patent implements dynamic context window adjustment where the model can adaptively modify its attention mechanism scope based on the input data characteristics and anomaly detection requirements. The context window size and attention span are adjusted dynamically during processing to focus on relevant temporal segments, enabling the model to identify root causes while maintaining manageable computational complexity through adaptive rather than fixed structures.
3Productivity
If system logs are processed without metadata extraction, then the processing pipeline is simple, but the speed of anomaly identification is slow
Solution Approach 1:
The patent performs preliminary metadata extraction and feature engineering before the actual anomaly detection process. By pre-processing system logs to extract meaningful metadata, normalize formats, and prepare feature representations in advance, the system reduces the computational burden during anomaly detection, thereby increasing identification speed without proportionally increasing overall pipeline complexity.
Data Source
AI summary
A method for managing an anomaly in a client includes: obtaining, by an analyzer, historical metadata (HM); obtaining, by the analyzer, an error description that is associated with the HM; analyzing, by the analyzer, the HM to generate a first data frame (DF); generating, by the analyzer, a second DF and a third DF based on the first DF, in which the second DF and the third DF are sent to an engine; generating, by the analyzer, a fourth DF based on the first DF and error description, in which the fourth DF is sent to the engine; tuning, by the engine, an anomaly detection model (ADM) to obtain a tuned ADM using: a first target parameter (TP) and the second DF; a second TP and the third DF; a third TP and the fourth DF; and initiating, by the engine, notification of an administrator about the tuned ADM.