Targeted Clustering With Vantage-Point Buckets for Malware Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustering algorithms form large, impure clusters that are difficult to craft signatures for, require constant parameter tuning, struggle with scalability and precision, and are computationally expensive, especially in big data environments like modern anti-malware systems.
Innovation Solution
A micro-clustering method using a 'Moon' vantage point and laser cutting strategy to form smaller, precise micro-clusters, followed by targeted clustering with multiple vantage points and bucketization to efficiently classify samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional clustering algorithms are used to group objects, then clusters are formed, but the clusters are large and impure, making signature authoring difficult
Solution Approach 1:
The patent divides the traditional clustering process into two stages: first forming large clusters using conventional algorithms, then segmenting these clusters into smaller micro-clusters with higher purity. This segmentation resolves the contradiction by maintaining the organizational benefit of clustering while achieving the precision needed for effective signature authoring.
Solution Approach 2:
The patent applies local quality by creating micro-clusters with locally optimized purity characteristics. Each micro-cluster is formed to have high internal homogeneity, ensuring that objects within the same micro-cluster share similar features, which directly improves measurement precision for signature authoring.
2Reliability
If traditional clustering algorithms are used, then clusters are formed, but constant parameter tuning is required
Solution Approach 1:
The patent performs preliminary action by pre-computing feature vectors and distances for all objects before the actual clustering process. This preparation work is done once and reused across multiple clustering operations, eliminating the need for constant parameter tuning and improving reliability without increasing operational complexity.
Solution Approach 2:
The system implements self-service through automatic parameter selection and adaptive clustering. The algorithm automatically adjusts parameters based on the data characteristics, eliminating manual tuning requirements while maintaining stable and reliable clustering results.
3Quantity of substance
If traditional clustering algorithms are used on big data, then clustering is performed, but computational cost is high
Solution Approach 1:
The patent segments the large dataset into smaller micro-clusters, which can be processed independently and in parallel. This segmentation reduces the computational burden on single processing units while maintaining the ability to handle big data, thereby lowering overall energy consumption.
Solution Approach 2:
The patent applies partial action by computing only the necessary feature vectors and distances required for clustering, rather than performing exhaustive analysis on all possible object combinations. This selective computation significantly reduces energy consumption while still achieving effective clustering of large datasets.
4Measurement precision
If clustering is performed to identify similar objects, then groups are formed, but precision and scalability are compromised
Solution Approach 1:
The patent performs preliminary computation of feature vectors and distance metrics before clustering. This pre-computation enables fast comparison and matching during the actual clustering process, achieving both high precision in object similarity measurement and high productivity through efficient processing.
Solution Approach 2:
The patent changes parameters by using different distance metrics and similarity thresholds for different stages of clustering. This parameter optimization allows the system to achieve high precision in identifying similar objects while maintaining scalability through adaptive parameter selection.
Data Source
Figure 1
Figure 2
Figure 3a
AI summary
There is disclosed a computer-implemented system and method of classifying a target sample, wherein the target sample is a computer object having a feature vector, the method comprising: creating n sorted containers, comprising sorting a universe of samples based on feature vector distances from the samples to n vantage points, wherein n is a positive integer; storing the n sorted containers to a computer memory; bucketizing the n sorted containers; for the n vantage points, selecting, from the n bucketized sorted containers, n meta-buckets of that the target sample belongs to; creating an intersection container, comprising samples that appear in all n meta-buckets; selecting, as a target cluster, samples from the intersection container that have a feature vector distance from the target sample less than a threshold; and acting on the target cluster.