Partial File Hashing for Malware Detection Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional anti-malware systems face computational burdens due to the need for full-file hashes, which can slow down client systems, especially as malware variants increase, making it inefficient to store and update large databases.
Innovation Solution
The method involves generating an initial partial fingerprint of a file on a client system, sending it to a server for initial malicious file checks, and only generating a full hash if the initial fingerprint indicates potential maliciousness, thereby minimizing resource-intensive operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full-file hashes are computed for each file before cloud lookup, then detection accuracy is improved, but computational burden and processing time increase
Solution Approach 1:
The file hashing process is segmented into two stages: first, a quick partial hash is computed on a subset of file bytes for initial screening; second, a complete full-file hash is computed only for files that fail the initial screening. This segmentation reduces the frequency of computationally expensive full hashing operations while maintaining detection accuracy.
Solution Approach 2:
The system performs partial hashing (computing hash on only a portion of the file) as a preliminary action before deciding whether to perform the complete hashing action. This partial action filters out many benign files quickly, reserving the excessive computational resource for only those files that require full verification.
2Reliability
If full-file hashes are computed for each file, then malicious file detection reliability is improved, but client system resource consumption increases
Solution Approach 1:
The detection process is divided into two segments: an initial low-resource screening phase using partial file hashing, and a secondary high-reliability verification phase using full-file hashing only for suspicious files. This segmentation allows the system to maintain detection reliability while significantly reducing average resource consumption.
Solution Approach 2:
The system applies partial hashing as a preliminary filter before committing to the resource-intensive full hashing operation. This ensures that full hashing (the reliable but expensive operation) is performed only when necessary, optimizing the balance between reliability and resource usage.
3Adaptability or versatility
If cloud-based malware databases are used, then detection capability is improved, but network dependency and latency increase
Solution Approach 1:
The system performs preliminary local processing (partial hashing and initial screening) before initiating network communication with the cloud database. This preliminary action prepares the data in advance, reducing the amount of back-and-forth network traffic required and minimizing the impact of network latency on overall detection time.
Data Source
AI summary
A computer-implemented method for detecting malicious files may include determining that a file on a client system may be subject to a security assessment, generating an initial fingerprint of the file, the generation of the initial fingerprint excluding at least part of the file, sending the initial fingerprint to a server and receiving a response from the server including an indication that the initial fingerprint matches at least one known malicious file but that the file from which the initial fingerprint was generated may not match the malicious file, generating an additional hash of the file on the client system based at least in part on the part of the file excluded in the generation of the initial fingerprint, sending the additional hash to the server, and receiving a response indicating that the file on the client system is malicious. Various other methods, systems, and computer-readable media are also disclosed.