Process Outlier Detection Using Software-Based Machine Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compromise assessment methods fail to effectively detect modern-day targeted threats in a timely manner due to insufficient data for training, high false positives, and ineffective clustering in diverse software environments, especially during reactive breach responses or vulnerability discoveries.
Innovation Solution
A method utilizing tf-idf techniques for clustering machines based on installed software and process information, combined with DBSCAN algorithm and weighted Jaccardian distance, to form initial clusters and iteratively detect outliers with adaptive frequency-based data collection, reducing false positives by grouping similar machines and confirming outliers across clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional signature-based detection methods are used, then detection accuracy for known malware is improved, but detection capability for modern targeted threats deteriorates
Solution Approach 1:
The patent transitions from signature-based detection to behavior-based detection by changing the detection parameters from static malware signatures to dynamic process behavior metrics. This includes monitoring process creation rates, network connection patterns, and system resource usage, allowing the system to detect targeted threats that lack known signatures while maintaining detection accuracy for known malware through anomaly detection
Solution Approach 2:
The patent replaces the mechanical signature-matching system with an intelligent anomaly detection system using machine learning algorithms. Instead of relying on pre-defined signatures, the system learns normal behavior patterns and automatically identifies deviations, enabling detection of both known and targeted threats without requiring manual signature updates
2Measurement precision
If supervised or unsupervised learning is applied to forensic database, then outlier detection capability is improved, but training time requirement increases to 20-30 days
Solution Approach 1:
The patent performs preliminary actions by continuously collecting and pre-processing forensic data in the background before compromise assessment is needed. The system maintains a ready-trained model through ongoing data collection and periodic retraining, so when a breach response or vulnerability discovery occurs, the analysis can begin immediately without requiring 20-30 days of data gathering and training
Solution Approach 2:
The patent implements continuous data collection and model training operations that run continuously in the background. This ensures the detection system is constantly improving and ready for immediate deployment, eliminating the need for lengthy training periods when compromise assessment is triggered, as the system maintains an perpetually updated understanding of normal behavior patterns
3Productivity
If simple frequency distribution of processes is used, then detection speed is improved, but false positive rate increases due to large software usage variance
Solution Approach 1:
The patent applies local quality by analyzing process frequency distribution within localized groups of similar machines rather than across the entire enterprise. By segmenting the environment into clusters with comparable software usage patterns, the system maintains high detection speed while reducing false positives, as each cluster's baseline reflects its specific operational context rather than enterprise-wide variance
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The invention concerns a method for detecting outliers in processes running in a group of machines; the method being carried out by a computer and comprising: - a clustering stage carried out at a first frequency and comprising the following steps: - fetching a list of software contained in all machines, - calculating tf-idf value for each installed software and for each machine, - performing clustering of the machines by applying a clustering algorithm and using Jaccardian weighted distance method between machines based on the tf-idf values, - a preliminary outliers detection stage carried out at a second frequency, the second frequency being greater than the first frequency, and said preliminary outliers detection stage comprising the following steps: - fetching information of processes running in the machines, - for each cluster, calculating tf-idf values for each process, if a tf-idf value is greater than a first predetermined threshold, the current process is considered as outlier, - for all clusters, calculating itf-idf value for each process considered as outlier, if a itf-idf value is lower than a second predetermined threshold, the current process is confirmed as outlier.