Cloud Security Polygraph Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analytics platforms face challenges in efficiently monitoring and detecting anomalies in cloud environments, particularly in datacenters, due to the complexity of network activities and the need for real-time data processing and security monitoring.
Innovation Solution
A data platform is configured to ingest data from cloud environments, process it in real-time, and generate polygraphs to model normal behaviors, allowing for the detection of deviations and anomalies through the use of agents, data aggregators, and microservices that create logical graphs of entity behaviors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time data processing is implemented to detect anomalies in cloud environments, then anomaly detection capability is improved, but system complexity increases
Solution Approach 1:
The system segments the cloud environment into multiple virtual machines, each equipped with its own agent that independently collects local data. This segmentation allows distributed processing, reducing the complexity burden on any single central processor while maintaining comprehensive real-time monitoring across the entire cloud infrastructure.
Solution Approach 2:
Data aggregators serve as intermediary components between the virtual machines and the central data processing system. These aggregators collect and pre-process data locally before transmitting to the central system, acting as intermediaries that reduce the volume and complexity of data requiring real-time processing while preserving anomaly detection capability.
2Reliability
If comprehensive security monitoring is deployed across cloud environments, then security coverage is improved, but data processing requirements increase
Solution Approach 1:
The system extracts and transmits only the most critical security-relevant data from the vast amount of cloud environment data. Agents selectively collect and filter data based on security parameters, extracting only essential information for anomaly detection while leaving the majority of routine operational data local, thus reducing processing volume while maintaining comprehensive security coverage.
Solution Approach 2:
Data aggregators perform preliminary data processing and filtering at intermediate stages before data reaches the central system. This preliminary action pre-processes and prioritizes data, reducing the volume of data requiring full analysis while ensuring all security-critical events are captured, thereby maintaining comprehensive monitoring with reduced processing requirements.
3Measurement precision
If polygraph generation is used to model normal behaviors, then anomaly detection precision is improved, but computational resources required increase
Solution Approach 1:
The system generates polygraphs locally at the virtual machine level rather than requiring centralized computation for all behavior modeling. Each virtual machine's agent creates local polygraphs representing its specific normal behavior patterns, which are then aggregated. This local quality approach reduces the computational resources required at the central system while maintaining high precision in anomaly detection through localized behavior modeling.
Data Source
AI summary
Learning from other cloud deployments to combat security threats, including: identifying, for at least a portion of a first cloud deployment, one or more additional cloud deployments to utilize for cross-customer learning; receiving information describing a security threat to one or more of the additional cloud deployments; receiving information describing configuration settings used to combat the security threat; and identifying, based on the information describing configuration settings used to combat the security threat, one or more configurations to adopt for the first cloud deployment.