Ensemble Clustering for Cybersecurity Noise Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning tools for cluster analysis in cybersecurity are susceptible to noise in datasets, leading to unstable and unreliable clustering results, which hinders the detection of abnormal computing actions and malicious activities in cloud and distributed computing environments.
Innovation Solution
A method involving the use of ensemble clustering methodologies that apply multiple clustering algorithms independently to a dataset without an initial cluster starting point, comparing clusters based on centroid locations and shared data points to identify and eliminate spurious clusters, thereby improving the reliability of cluster detection and security operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing machine learning tools for cluster analysis are used, then clustering can be performed on security data, but the results are unstable and unreliable due to noise susceptibility
Solution Approach 1:
The patent combines multiple independent clustering methodologies into an ensemble approach. Each clustering method processes the security data independently, and their results are merged through comparison of centroid locations and shared data points. This combination allows the system to identify consistent patterns across different methods while filtering out noise-specific to individual algorithms, thereby improving reliability and reducing noise susceptibility.
2Reliability
If multiple clustering methodologies are applied independently, then spurious clusters can be identified and eliminated, but the computational complexity and time required increase
Solution Approach 1:
The patent performs preliminary actions by applying multiple clustering methodologies independently before the final cluster identification stage. Each clustering method pre-processes the data and generates initial cluster assignments. These preliminary results are then compared to identify spurious clusters, allowing the system to eliminate unreliable clusters early in the process rather than performing extensive analysis on all clusters generated by a single method.
Solution Approach 2:
The patent discards spurious clusters identified through the ensemble comparison process. By comparing centroid locations and shared data points across multiple clustering methods, the system identifies and discards clusters that are inconsistent or noise-driven. This selective discarding recovers computational resources and focuses subsequent security operations only on validated, reliable clusters, improving efficiency despite the initial increased computational load.
3Measurement precision
If cluster comparison based on centroid locations and shared data points is performed, then spurious clusters can be isolated, but the processing complexity increases
Solution Approach 1:
The patent applies local quality by performing comparisons at specific local features of clusters rather than analyzing entire cluster structures globally. The system compares centroid locations (a specific point representation of each cluster) and shared data points (specific member overlaps) between clusters from different methodologies. This localized comparison approach achieves high measurement precision in identifying spurious clusters while keeping the algorithm complexity manageable by focusing on key discriminative features rather than comprehensive cluster analysis.
Data Source
AI summary
Disclosed herein are methods, systems, and processes to detect valid clusters and eliminate spurious clusters in cybersecurity-based computing environments. A cluster detection and elimination model is trained by accessing a dataset with raw data that includes data points associated with computing devices in a network and applying two or more different clustering methodologies independently to the dataset. The resulting cluster detection and elimination model is used to compare two or more clusters to determine whether a cluster from one clustering methodology matches another cluster from another clustering methodology based on centroid locations and shared data points.