Malicious Software Clustering via TLSH Feature Compression

The patent segments the feature extraction process into two distinct phases: static analysis (extracting features from binary code, import tables, and metadata) and dynamic analysis (extracting features from runtime behavior, API calls, and system interactions). This segmentation allows comprehensive feature extraction while managing computational complexity by processing different feature types independently and combining results at the clustering stage.

The patent transforms the high-dimensional feature space into a lower-dimensional representation by selecting the top-k most discriminative features based on statistical analysis and information theory metrics. This parameter change in dimensionality reduces the computational burden of clustering operations while preserving the most informative features that contribute to clustering accuracy.

The patent employs multiple lightweight clustering algorithms (including density-based and hierarchical methods) that can be executed rapidly and discarded, replacing the need for a single complex algorithm. This approach enables the system to try different clustering strategies, identify noise through consensus or outlier detection across multiple runs, and handle arbitrary cluster shapes without committing to one expensive, complex algorithmic framework.

The patent implements automated feature selection through self-service mechanisms that evaluate feature importance using information theory metrics (such as entropy, mutual information) and statistical tests. The system automatically identifies and removes redundant or low-quality features without human intervention, maintaining feature quality while achieving full automation of the feature engineering pipeline.