Compiler-Injected Branch Auditor for Control Flow Hijacking Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for protecting branching control flow in computer programs are ineffective against reverse engineering attacks, as once a jump is hijacked, a reverse engineer can undo changes, making malicious paths undetectable.
Innovation Solution
A compiler-injected branch auditor is used to select and audit critical branches by inserting calls through an API, recording and obfuscating branch paths and parameters, and invoking a branch auditor asynchronously to detect hijacking, which can perform security operations such as terminating the program or erasing data if the path is invalid.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional methods verify instructions and data have not been tampered with, then branch protection is attempted, but the reverse engineer can undo changes and make malicious paths undetectable
Solution Approach 1:
The patent applies preliminary action by recording the expected branch path and parameters before the program executes. The branch auditor captures the intended control flow path and compares it against the actual executed path, allowing detection of hijacking attempts before they can complete their malicious operations. This proactive recording mechanism ensures that even if a reverse engineer attempts to undo changes, the original expected path is preserved for comparison.
Solution Approach 2:
The patent implements feedback by continuously monitoring the actual branch execution path and comparing it against the recorded expected path. The branch auditor provides real-time feedback on whether the program is executing along the intended control flow, enabling immediate detection of deviations caused by reverse engineering attacks or malicious hijacking attempts.
2Difficulty of detecting and measuring
If the branch auditor records and compares branch paths asynchronously, then detection capability is improved, but the complexity of the system increases
Solution Approach 1:
The patent applies the extraction principle by isolating the branch auditing functionality into a separate, dedicated component called the branch auditor. This extracted module independently records expected paths, monitors actual execution, and performs comparisons without interfering with the main program logic. The asynchronous operation is handled within this isolated auditor component, reducing the complexity burden on the overall system while maintaining strong detection capabilities.
Data Source
AI summary
A branch auditing system can be automatically injected into a computer program, in one embodiment, in response to a programming call provided in source code by a programmer who has selected a particular branch, in a set of possible branches, for auditing. The branch auditing system can record, in an obfuscated data structure, a path taken at the particular branch and the parameters associated with the branch and later an auditor can determine whether the path taken was valid, and if the path taken was invalid, operations can be performed to protect the program, system and/or user.