Cloud Spam Detection via Locality-Sensitive Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for identifying spam and malware in communication networks are inefficient and lack centralized monitoring and control, leading to inadequate detection and false positives.
Innovation Solution
An in-the-cloud apparatus and method using locality-sensitive hashes to identify spam and malware by processing electronic messages, calculating hashes, embedding them into queries, and sending them to an analysis system for detection and grouping, with a centralized data structure for signature creation and matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If decentralized anti-spam software is deployed in host computers, then detection capability is provided at individual devices, but centralized monitoring and control of spam outbreaks is lost
Solution Approach 1:
The patent combines decentralized detection capabilities with centralized monitoring by merging host-based hash verification with a centralized server that collects hash data, identifies outbreaks, and distributes signatures. This integration allows both local detection and global coordination to function together in a unified system.
Solution Approach 2:
The patent implements a nested architecture where host computers contain hash-based detection mechanisms that are themselves nested within a larger centralized outbreak management system. The host-level verification is embedded within the broader cloud-based coordination framework, allowing multiple levels of detection to operate hierarchically.
2Reliability
If traditional DNS blacklist/whitelist methods are used, then known spam sources can be blocked, but false positive identifications occur and detection efficiency is insufficient
Solution Approach 1:
The patent uses hash codes as simplified copies or representations of complete spam messages. Instead of analyzing entire messages or maintaining lists of specific spam content, the system creates compact hash representations that can be efficiently distributed and verified across the network, reducing false positives while maintaining detection accuracy.
Solution Approach 2:
The patent changes the fundamental parameter used for spam detection from content-based analysis (DNS blacklists/whitelists) to hash-based identification. This parameter transformation enables more precise matching and reduces false positives by focusing on unique cryptographic representations rather than content analysis.
3Measurement precision
If comprehensive spam detection is performed, then detection accuracy improves, but processing time and system resources increase
Solution Approach 1:
The patent extracts only the essential identifying feature of spam messages (the hash code) and separates it from the full message content. By taking out just the critical verification element and processing only that, the system achieves accurate detection without the time cost of analyzing complete messages.
Solution Approach 2:
The patent performs preliminary hashing of spam messages before they need to be fully analyzed or acted upon. By pre-computing and distributing hash codes in advance, the system enables rapid verification at host level without requiring time-consuming content analysis when actual detection is needed.
Data Source
AI summary
One embodiment relates to an apparatus for in-the-cloud identification of spam and/or malware. The apparatus includes computer-readable code configured to be executed by the processor so as to receive queries, the queries including hash values embedded therein. The apparatus further includes computer-readable code configured to be executed by the processor so as to detect a group of hash codes which are similar and to identify the group as corresponding to an undesirable network outbreak. Another embodiment relates to an apparatus for in-the-cloud detection of spam and/or malware. The apparatus includes computer-readable code configured to be executed by the processor so as to receive an electronic message, calculate a locality-sensitive hash based on the message, embed the locality-sensitive hash into a query, and send the query to a central analysis system via a network interface. Other embodiments, aspects and features are also disclosed.