Cloud Data Platform Polygraph Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analytics platforms face challenges in efficiently monitoring and detecting anomalies in cloud environments, particularly in datacenters, due to the complexity of network activities and the need for real-time data processing and security monitoring.
Innovation Solution
A data platform is configured to ingest data from cloud environments, process it using agents deployed on compute assets, and generate polygraphs to model normal behaviors, allowing for the detection of deviations and anomalies in real-time, utilizing a combination of data ingestion resources, processing resources, and user interface resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time data processing and security monitoring are implemented in cloud environments, then anomaly detection capability and security monitoring are improved, but data complexity and processing requirements increase
Solution Approach 1:
The system segments data processing by deploying agents on individual compute assets to collect local data, separating the complexity of data collection from central processing. The polygraph model segments anomaly detection into pattern matching operations that can be performed efficiently on structured behavioral data rather than raw complex data
Solution Approach 2:
The polygraph model acts as an intermediary that transforms complex network activity data into simplified behavioral patterns and expectations. This mediator structure allows the system to process complex cloud environment data through a standardized framework of normal behavior models, reducing processing complexity while maintaining detection accuracy
2Productivity
If agents are deployed on compute assets to collect and report information, then real-time monitoring capability is improved, but system resource consumption increases
Solution Approach 1:
The agents perform self-service by autonomously collecting, filtering, and reporting only the information needed for anomaly detection. The polygraph model enables agents to self-determine what constitutes normal behavior and automatically identify deviations without requiring continuous external processing resources
Solution Approach 2:
The system applies partial action by having agents collect and process only the specific data elements necessary for security monitoring and anomaly detection, rather than capturing all possible system data. This selective data collection reduces resource consumption while maintaining effective monitoring capability
Data Source
AI summary
Identifying encountered and unencountered conditions in software applications, including: collecting, for an executing application, information describing the usage of the application, including: receiving, from one or more tracepoints inserted into the application, a first portion of the information describing a state of the application during execution; identifying, based on the information, one or more unencountered conditions that the application is configured to handle; and presenting information describing the unencountered conditions that the application is configured to handle.