Edge Anomaly Detection via Log Pattern Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection in computing environments is hindered by the high volume of log data required for pattern extraction, which is costly to transport and process, leading to operational overhead and inefficiencies in detecting anomalous conditions.
Innovation Solution
An anomaly service identifies dominant patterns in log data from edge nodes, extracts non-anomalous blocks, and generates anomaly vectors to detect anomalies at their source, reducing bandwidth usage and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log data is transported from edge nodes to centralized server for pattern extraction, then anomaly detection capability is improved, but bandwidth consumption and operational overhead increase significantly
Solution Approach 1:
Instead of extracting patterns from all log data at a centralized server, the patent inverts the approach by distributing pre-trained anomaly detection models to edge nodes. These models extract only anomaly-related patterns locally, sending minimal anomaly indicators to the server rather than transporting entire log datasets.
Solution Approach 2:
The patent segments the anomaly detection system into distributed components: local pattern extraction at edge nodes, selective anomaly identification, and centralized model training. This segmentation allows each component to operate independently with reduced data transmission requirements.
2Reliability
If pattern extraction is performed on all log data, then comprehensive anomaly detection is achieved, but computational resources required exceed allocated capacity
Solution Approach 1:
The patent extracts only the essential anomaly-related patterns from log data using pre-trained models at edge nodes, rather than performing comprehensive pattern extraction on all data. This extraction approach identifies and processes only the critical subset of data containing anomaly indicators.
Solution Approach 2:
Anomaly detection models are pre-trained on historical data before deployment to edge nodes. This preliminary action enables the models to perform efficient real-time anomaly detection without requiring extensive computational resources during operational data processing.
3Measurement precision
If extensive log data is processed to extract patterns, then detection accuracy improves, but time required for pattern extraction becomes prohibitively long
Solution Approach 1:
Pattern extraction models are pre-trained on historical log data in advance, creating ready-to-use anomaly detection capabilities. This preliminary action eliminates the need for time-consuming pattern extraction during real-time operations, enabling immediate anomaly detection when deployed at edge nodes.
Data Source
AI summary
An anomaly service receives log data from nodes in a computing environment, which includes a sequence of information indicative of log messages produced by the nodes. The anomaly service identifies dominant patterns in the sequence of information that are representative of non-anomalous blocks of the log messages. Having identified the dominant patterns, the service is able to extract the non-anomalous blocks from the log data to reveal anomalous blocks that do not fit the dominant patterns. The service may then generate anomaly vectors based on the anomalous blocks, which can be distributed to the nodes to detect anomalies.