Virtual Machine Introspection for Encrypted Traffic Decryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing use of encryption for malicious purposes, such as concealing malware activities, complicates anti-malware operations, especially in virtualized environments where hardware virtualization reduces investment and operating costs but escalates security threats, necessitating efficient anti-malware systems to protect virtual machines from malware.
Innovation Solution
A system comprising a client system with a hardware processor and memory, where the processor executes a virtual machine and an introspection engine to identify changed memory pages during communication sessions, allowing the decryption engine to decrypt encrypted payloads outside the virtual machine, collaborating with a security server to intercept and decrypt potentially malicious network traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption is used to protect communications, then security is improved, but the ability to detect malicious activities deteriorates
Solution Approach 1:
The patent introduces a decryption engine as an intermediary component that sits between the encrypted communication channels and the malware detection system. This decryption engine uses obtained decryption keys to convert encrypted malware communications into readable format, enabling the malware detection engine to analyze the content without breaking the security of legitimate encrypted communications. The intermediary approach allows selective decryption only when decryption keys are successfully obtained, thus maintaining security while enabling detection.
2Ease of manufacture
If hardware virtualization is used to reduce costs, then investment and operating costs are reduced, but security threats escalate
Solution Approach 1:
The patent segments the security protection function into separate modular components: a decryption key obtaining module, a decryption engine, and a malware detection engine. This segmentation allows the security system to be selectively applied to individual virtual machines or specific communication sessions within the virtualized environment. Each component can be independently configured and activated based on security requirements, enabling cost-effective deployment where full decryption capabilities are applied only where necessary rather than universally across all virtual machines.
3Difficulty of detecting and measuring
If decryption is performed on all encrypted traffic, then malicious activities can be detected, but computational effort increases
Solution Approach 1:
The patent implements partial decryption action by selectively decrypting only those encrypted communications for which decryption keys have been successfully obtained. The system does not attempt to decrypt all encrypted traffic universally, but rather applies decryption on a case-by-case basis when the decryption key obtaining module successfully retrieves the necessary keys. This partial action approach significantly reduces computational effort compared to universal decryption while still maintaining the capability to detect malicious activities in decrypted traffic.
Data Source
Figure 1
Figure 2-A~2-B
Figure 3
AI summary
Described systems and methods enable a decryption of encrypted communication between a client system and a remote party, for applications such as detection and analysis of malicious software, intrusion detection, and surveillance, among others. The client system executes a virtual machine and an introspection engine outside the virtual machine. The introspection engine is configured to identify memory pages whose contents have changed between a first session event (e.g., a ServerHello message) and a second session event (e.g., a ClientFinished message). The respective memory pages are likely to contain encryption key material for the respective communication session. A decryption engine may then attempt to decrypt an encrypted payload of the respective communication session using information derived from the content of the identified memory pages.