Fuzzy Hashing for Malware Variant Classification

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional malware classification techniques fail to accurately classify malware objects that share similarities but have small differences, leading to inaccurate and inefficient classification, especially for zero-day threats.

Innovation Solution

The use of fuzzy hashing to compare the behaviors of suspect objects with previously classified objects, allowing for similarity matching rather than exact matches, and machine learning to determine associations with clusters for classification.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional direct comparison techniques are used to classify malware objects, then exact matches can be detected, but similar malware objects with small differences cannot be properly classified

Engineering Contradiction:
Improveclassification accuracyVSAvoidability to detect similar malware variants
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent transforms the classification approach by changing the parameter from exact byte-by-byte comparison to fuzzy hashing with similarity thresholds. Instead of requiring identical matches, the system uses hash values with configurable similarity thresholds to identify malware variants that share common characteristics while allowing for modifications, thereby improving detection of similar malware objects

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent replaces the mechanical direct comparison system with a fuzzy hashing mechanism. Rather than systematically comparing each byte of malware objects, the system substitutes this with hash value generation and similarity scoring, enabling efficient detection of similar malware while accommodating variations in the code structure

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If fuzzy hashing is used to allow similarity matching, then detection of similar malware variants improves, but processing complexity increases

Engineering Contradiction:
Improveability to detect similar malware variantsVSAvoidprocessing complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent creates simplified representations (hashes) of malware objects that capture essential characteristics without requiring full object storage or comparison. By copying only the critical behavioral patterns into hash values, the system maintains detection capability while significantly reducing processing complexity and memory requirements

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system performs partial comparison through fuzzy hashing rather than exhaustive byte-by-byte analysis. By focusing computational effort on generating and comparing hash values rather than analyzing entire malware objects, the system achieves adequate detection accuracy with reduced processing complexity

Inventive Principle:
Principle #16Partial or excessive action

3Productivity

If traditional hashing is used for exact matching, then processing speed is maintained, but classification accuracy for variant malware decreases

Engineering Contradiction:
Improveprocessing speedVSAvoidclassification accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent changes the hashing parameter from exact-match oriented to similarity-oriented by implementing fuzzy hashing algorithms. This allows the system to maintain fast processing speeds through hash value comparison while improving classification accuracy by detecting similar malware variants that traditional exact-match hashing would miss

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS10218740B1Fuzzy hash of behavioral results
Publication Date: 2019.02.26 MANDIANT LLC
  • US10218740B1 patent drawing
  • US10218740B1 patent drawing
  • US10218740B1 patent drawing

AI summary

A computerized method for classifying objects in a malware system is described. The method includes detecting behaviors of an object for classification after processing of the object has begun. Data associated with the detected behaviors is collected, and a fuzzy hash for the received object is generated. The generation of the fuzzy hash may include (i) removing a portion of the data associated with the detected behaviors, and (ii) performing a hash operation on a remaining portion of the data associated with the detected behaviors. Thereafter, the fuzzy hash for the received object is compared to a fuzzy hash of an object in a preexisting cluster to generate a similarity measure. The received object is associated with the preexisting cluster in response to determining that the similarity measure is above a predefined threshold value. Thereafter, the results are reported.