Data Platform Community Policy Generation for Cloud Anomaly Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data security and analytics systems struggle to efficiently monitor and manage large-scale cloud environments, lacking effective tools for real-time anomaly detection and compliance monitoring across diverse compute assets.
Innovation Solution
A data platform that integrates data ingestion, processing, and user interface resources to monitor and manage cloud environments, utilizing agents to collect data from compute assets and generate polygraphs for anomaly detection and compliance monitoring, with community-based policy generation for customized analytics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing data security and analytics systems are used to monitor cloud environments, then basic monitoring capabilities are provided, but real-time anomaly detection and compliance monitoring across diverse compute assets cannot be effectively achieved
Solution Approach 1:
The system segments the monitoring function by deploying agents on individual compute assets to collect local data, then aggregates this data at centralized processing components. This segmentation enables distributed real-time monitoring across diverse compute assets while maintaining centralized control for anomaly detection and compliance monitoring, resolving the contradiction between reliability and productivity.
Solution Approach 2:
The patent introduces polygraphs as intermediary data structures that normalize and represent computed asset data in a unified format. These polygraphs serve as mediators between diverse compute assets and the analysis system, enabling effective real-time anomaly detection and compliance monitoring across heterogeneous environments without requiring system redesign.
2Ease of operation
If centralized monitoring systems are used, then compliance monitoring is simplified, but real-time anomaly detection across diverse compute assets becomes inefficient
Solution Approach 1:
The system divides monitoring into two segments: local agents on compute assets that perform real-time data collection and preliminary processing, and centralized components that handle compliance monitoring. This segmentation enables fast local anomaly detection while maintaining simplified centralized compliance oversight, resolving the contradiction between ease of operation and speed.
Solution Approach 2:
Agents perform preliminary actions by collecting and pre-processing data locally at compute assets before transmitting to centralized systems. This preliminary action reduces the volume of data requiring centralized processing, enabling both fast local anomaly detection and simplified compliance monitoring through aggregated, pre-processed information.
3Adaptability or versatility
If customized analytics policies are implemented for each compute asset, then adaptability to diverse environments improves, but system complexity increases
Solution Approach 1:
The patent implements a universal policy framework where compute asset owners can define customized analytics policies using a standardized polygraph-based language. This universal interface allows diverse compute assets to have asset-specific policies while using the same underlying processing mechanisms, improving adaptability without proportionally increasing system complexity.
Solution Approach 2:
The system uses template-based policy definitions where common monitoring patterns can be copied and reused across different compute assets. This copying mechanism enables rapid customization for diverse environments while avoiding the complexity of creating unique policies from scratch, as templates can be instantiated and adapted as needed.
Data Source
AI summary
A disclosed data platform may be configured to access a custom policy developed by a particular client entity that uses the data platform. The custom policy may invoke a query that targets a target dataset ingested from a cloud environment and stored in a data store. The data platform may determine that the custom policy is likely to be of value to one or more other client entities, besides the particular client entity, that also use the data platform. In response to the determining that the custom policy is likely to be of value to the one or more other client entities, the data platform may generate a community policy based on the custom policy. The community policy may be available for use by the one or more other client entities. Corresponding methods, systems, and products are also disclosed.