Kubernetes Anomaly Detection via Role-Based ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network security solutions for Kubernetes clusters rely on simple static policies, which are prone to false positives and fail to address unknown threats or vulnerabilities, thereby lacking confidence in proactive action.
Innovation Solution
A machine-learning based approach that learns normal behavior for various user roles within a Kubernetes cluster using data from the Kubernetes API server audit log augmented with external data, enabling proactive action based on anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If simple static policies are used for security monitoring, then device complexity is reduced, but reliability of threat detection deteriorates due to false positives and inability to address unknown threats
Solution Approach 1:
The patent transforms static security policies into dynamic, adaptive policies by implementing a machine learning model that continuously learns from audit logs and automatically updates anomaly detection thresholds. The system evolves its understanding of normal behavior patterns over time, allowing it to adapt to new threats and reduce false positives without requiring manual policy updates.
Solution Approach 2:
The security system performs self-learning and self-adjustment by automatically analyzing audit logs, identifying normal behavior patterns, and updating anomaly detection criteria without external intervention. The machine learning model continuously trains on incoming data, enabling the system to improve its detection capabilities autonomously while maintaining operational simplicity.
2Measurement precision
If machine-learning based anomaly detection is implemented, then reliability of suspicious activity detection improves, but device complexity increases due to ML model training and data processing requirements
Solution Approach 1:
The patent segments the security monitoring system into distinct functional modules: audit log collection, data preprocessing, machine learning model training, anomaly detection, and alert generation. This modular architecture allows each component to be optimized independently and facilitates easier maintenance and deployment while achieving high detection precision.
Solution Approach 2:
The system introduces an intermediary layer between raw audit logs and security decisions, consisting of feature extraction and preprocessing modules. This intermediary transforms complex, unstructured log data into standardized features that the machine learning model can efficiently process, reducing the computational burden while maintaining high detection accuracy.
3Measurement precision
If audit logs are augmented with external data such as user roles and anomaly thresholds, then measurement precision of behavior analysis improves, but loss of information processing increases due to additional data handling requirements
Solution Approach 1:
The patent applies local quality by selectively augmenting audit logs with external data only where relevant to specific user roles and contexts. Rather than uniformly adding all possible external data, the system tailors the augmentation to each user's role, resource access patterns, and historical behavior, optimizing the signal-to-noise ratio and reducing unnecessary data processing overhead.
Data Source
AI summary
Systems and methods are provided for learning normal behavior for user roles of an application running within a cluster of container orchestration platform and based thereon proactively taking action responsive to suspicious events. According to one embodiment, an event data stream is created by an API server of the cluster. The data for each event includes information regarding a request made to an API exposed by the API server with which the event is associated and a user of the application by which the event was initiated. The data is augmented with a role associated with the user and an anomaly threshold for the role. Normal behavior is learned by an ML algorithm of respective user roles by processing the augmented data. When an anomaly score associated with a particular event is output by the ML algorithm that exceeds the anomaly threshold, a predefined or configurable action may be triggered.