Inverted Index Byte Sequence Search for Malware Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing malware detection methods are inefficient due to the disguising and obfuscation techniques employed by malware authors, making it time-consuming to determine if a program is malicious.
Innovation Solution
A system utilizing a binary search engine and an inverted index to search for byte sequences in files, generating signatures and fuzzy hashes, and evaluating search results using logical operators to efficiently identify malware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional malware detection methods are used, then detection accuracy can be maintained, but detection time increases significantly due to obfuscation techniques
Solution Approach 1:
The patent segments malware detection into multiple stages: first constructing an inverted index of byte sequences from clean files, then using this index to rapidly filter and identify suspicious byte patterns in target files. This segmentation allows the system to avoid comprehensive analysis of all files while maintaining detection accuracy, thereby reducing detection time despite obfuscation techniques
Solution Approach 2:
The system performs preliminary action by pre-processing clean files to build an inverted index of byte sequences before actual malware detection occurs. This pre-computed index enables rapid lookup and comparison during detection, allowing the system to quickly identify malicious patterns without performing time-consuming analysis on every file, thus resolving the contradiction between detection speed and detection difficulty
2Reliability
If comprehensive file analysis is performed to ensure accurate malware detection, then detection reliability improves, but processing speed decreases
Solution Approach 1:
The patent applies local quality by focusing analysis only on specific byte sequences that are likely to indicate malware, rather than performing comprehensive analysis of entire files. The inverted index enables the system to identify and analyze only relevant local portions of files containing suspicious byte patterns, maintaining detection reliability while significantly improving processing speed by avoiding unnecessary analysis of clean file regions
Data Source
Figure 1a
Figure 1b
Figure 1c
AI summary
Techniques for searching an inverted index associating byte sequences of a fixed length and files that contain those byte sequences are described herein. Byte sequences comprising a search query are determined and searched in the inverted index, and an intersection of the results is determined and returned as a response to the search query. Further, search queries in the form of expressions including search terms and logical operators are searched in the inverted index and evaluated using a syntax tree constructed based on the logical operators. Also, byte sequences comprising a file are searched in the inverted index and results of the search are used to generate signatures and fuzzy hashes.