Targeted Scanning for Encrypted RAT Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing use of encryption in network traffic makes it challenging for existing techniques like deep packet inspection to detect remote administration Trojans (RATs) and other malware, as these threats are concealed within encrypted communications, making it difficult to identify and prevent malicious data exfiltration.
Innovation Solution
A security service classifies traffic telemetry data to identify potential RAT activity, constructs targeted scan messages to elicit specific server responses, and analyzes these responses to determine if an endpoint device is infected, using machine learning and pattern recognition to differentiate between benign and malicious traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If deep packet inspection is used to detect malware, then detection capability is improved for unencrypted traffic, but detection capability deteriorates for encrypted traffic
Solution Approach 1:
The patent introduces an intermediary scanning mechanism that operates at the network layer to detect RAT characteristics without requiring decryption of the payload. The scanner sends probe packets and analyzes response patterns (timing, sequence, content) to identify malware behavior, serving as a mediator between network monitoring and encrypted traffic analysis.
Solution Approach 2:
The patent replaces the mechanical deep packet inspection approach (which requires payload decryption) with a statistical pattern recognition system that analyzes traffic behavior characteristics. Instead of directly examining encrypted content, the system substitutes analysis of meta-data patterns, timing information, and communication protocols to infer malware presence.
2Measurement precision
If targeted scanning is performed to detect RATs, then detection precision is improved, but network traffic disruption increases
Solution Approach 1:
The patent applies local quality by tailoring scan messages to specific detected RAT types and targeting only the relevant communication channels. Instead of blanket scanning, the system customizes probe packets based on the specific RAT variant detected (e.g., different scan patterns for Emotet vs. TrickBot), thereby improving precision while minimizing unnecessary traffic generation.
Solution Approach 2:
The patent employs partial action by performing scanning only on traffic flows that exhibit suspicious characteristics identified by preliminary analysis. The system selectively applies deep inspection to a subset of potential threats rather than all traffic, reducing overall traffic disruption while maintaining high detection precision for actual malware.
Data Source
AI summary
In one embodiment, a security service classifies traffic telemetry data for traffic between an endpoint device and a server as potentially associated with a particular type of remote access Trojan (RAT). The security service constructs a scan message to elicit a type of server response associated with the particular type of RAT. The security service obtains a server response from the server, by sending the constructed scan message to the server. The security service determines whether the endpoint device is infected with the particular type of RAT, by validating whether the server response from the server matches the type of server response associated with the particular type of RAT.