Control Flow Anomaly Detection for Process Hijacking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current security products face challenges in detecting process hijacking attacks, particularly due to their computational burdensomeness and inability to handle encrypted applications, as well as their reliance on compiler cooperation and signature-based behavioral detection methods.
Innovation Solution
The implementation of an anomaly detection system that employs performance telemetry information and machine learning techniques, utilizing normalization techniques to reduce noise and complexity in target instruction pointer address information, enabling dynamic learning and time-series analysis independent of source code encryption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If signature-based behavioral detection methods are used to detect process hijacking, then detection capability is improved, but computational burden increases and the system cannot handle encrypted applications
Solution Approach 1:
The patent replaces traditional signature-based behavioral detection (mechanical analysis of code patterns) with machine learning-based anomaly detection. The system uses trained models to automatically identify hijacking attempts by comparing actual control flow against expected patterns, eliminating the need for manual signature databases and reducing computational complexity while maintaining detection accuracy even with encrypted applications
Solution Approach 2:
The patent changes the detection parameters from static signature matching to dynamic anomaly scoring. By using machine learning models that evaluate multiple control flow features simultaneously and generate anomaly scores, the system adapts to different hijacking techniques without requiring updates to signature databases, thereby reducing computational burden while improving reliability
2Reliability
If control flow analysis is performed on encrypted applications, then detection coverage is improved, but the inability to decode encrypted code prevents accurate analysis
Solution Approach 1:
The patent introduces machine learning models as intermediaries between the encrypted code and the detection system. These models analyze control flow patterns, register operations, and memory access sequences without requiring decryption of the actual code. The intermediary translates encrypted binary behavior into interpretable features that reveal hijacking attempts while preserving detection coverage
Solution Approach 2:
The patent substitutes traditional static code analysis (which requires source code accessibility) with dynamic behavior analysis using machine learning. The system observes runtime control flow characteristics, instruction patterns, and execution sequences to detect anomalies, enabling detection coverage for encrypted applications while avoiding the need to decode or interpret the actual source code
3Measurement precision
If normalization techniques are applied to reduce noise in target instruction pointer address information, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies normalization techniques as preliminary processing steps before the main anomaly detection. By pre-computing normalized control flow representations and storing them in optimized data structures, the system reduces noise and complexity in advance, enabling faster real-time detection without sacrificing accuracy. The heavy lifting of normalization is performed once during data preparation rather than repeatedly during detection
Solution Approach 2:
The patent extracts and separates the normalization process from the core detection algorithm. By isolating the computationally intensive normalization steps into a dedicated preprocessing module, the system can optimize this separate component without impacting the speed of the main detection engine. The extracted normalized features are then efficiently consumed by the anomaly detection model, maintaining both accuracy and processing speed
Data Source
AI summary
Methods, systems, articles of manufacture and apparatus to detect process hijacking are disclosed herein. An example apparatus to detect control flow anomalies includes a parsing engine to compare a target instruction pointer (TIP) address to a dynamic link library (DLL) module list, and in response to detecting a match of the TIP address to a DLL in the DLL module list, set a first portion of a normalized TIP address to a value equal to an identifier of the DLL. The example apparatus disclosed herein also includes a DLL entry point analyzer to set a second portion of the normalized TIP address based on a comparison between the TIP address and an entry point of the DLL, and a model compliance engine to generate a flow validity decision based on a comparison between (a) the first and second portion of the normalized TIP address and (b) a control flow integrity model.