Control Transfer Termination Instructions for ROP Attack Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques to defend against return-oriented programming (ROP) attacks are ineffective, allowing attackers to execute unintended sequences of instructions by exploiting control transfer vulnerabilities in the x86 instruction set architecture.
Innovation Solution
The introduction of control transfer termination (CTT) instructions that enforce valid control transfer points, preventing unintended gadgets from being executed by ensuring that control transfers occur only to programmer-intended locations, using mode-specific opcodes that function as no-operations within the processor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If control transfer instructions are allowed to execute without restriction, then program flexibility and execution speed are maintained, but security vulnerabilities arise allowing ROP attacks
Solution Approach 1:
The patent applies preliminary action by inserting CTT instructions at compile time into specific control transfer locations (function prologues, epilogues, exception handlers). These instructions are prepared in advance to enforce security policies, so when ROP attacks attempt to execute unintended control transfers, the pre-placed CTT instructions block them. This resolves the contradiction by establishing security constraints beforehand without requiring complex runtime analysis or validation mechanisms.
2Reliability
If CTT instructions are inserted at all control transfer points, then security against ROP attacks is enhanced, but program execution overhead increases
Solution Approach 1:
The patent applies local quality by selectively placing CTT instructions only at specific critical control transfer locations identified through compiler analysis, rather than uniformly at all control transfer points. The compiler identifies locations such as function prologues, epilogues, and exception handlers where control transfers occur, and inserts CTT instructions only at these locations. This selective approach maintains security against ROP attacks while minimizing the number of CTT instructions executed, thereby reducing performance overhead.
3Reliability
If comprehensive control transfer tracking is implemented, then all unintended control transfers are detected, but processing overhead and energy consumption increase
Solution Approach 1:
The patent applies the taking out principle by extracting and eliminating unnecessary control transfer tracking from the system. Instead of implementing comprehensive tracking of all control transfers, the approach removes tracking requirements for control transfers that are already validated by the presence of CTT instructions at critical locations. The compiler analysis identifies which control transfers need enforcement, and only those are tracked and validated by CTT instructions, while other control transfers proceed without overhead.
Data Source
AI summary
A processor includes an execution unit and a processing logic operatively coupled to the execution unit, the processing logic to: enter a first execution state; responsive to executing a control transfer instruction having a pre-defined subcode, remain in the first execution state; responsive to executing a control transfer instruction not having the pre-defined subcode, transition into a second execution state; and responsive to determining, in the second execution state, that a next instruction to be executed differs from an ENDBRANCH instruction, raise an execution exception.


