Function Entry Exit Hash Verification for ROP Attack Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems face challenges in securing information handling and data processing systems from unauthorized access and attacks such as Return Oriented Programming (ROP) and Data Oriented Programming (DOP), which can hijack control flow and access sensitive data.
Innovation Solution
A cryptographic hash is computed using a secret key stored in a special purpose register, with the hash values checked upon function entry and exit to ensure integrity, and a trap interrupt is triggered if the values do not match, transferring control to the operating system to prevent malicious attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hash verification is implemented at function entry and exit, then security against ROP and DOP attacks is improved, but processing time and computational overhead increase
Solution Approach 1:
The patent computes and stores hash values of stack pointer and return address at function entry before the function executes. This preliminary action allows the hash to be ready for verification at function exit, avoiding the need to recompute it and reducing the time penalty at the critical exit point where security verification is most needed.
Solution Approach 2:
The patent replaces traditional mechanical security mechanisms (such as stack canaries or manual validation) with cryptographic hash verification. By using hash functions to verify the integrity of stack pointer and return address, the system achieves stronger security guarantees with automated verification that is more efficient than manual security checks.
2Reliability
If secret key is stored in special purpose register accessible only at higher privilege level, then security against unauthorized access is improved, but system complexity increases
Solution Approach 1:
The patent segments the secret key storage from general-purpose registers by placing it in a special-purpose register that is only accessible at higher privilege levels. This segmentation isolates the critical security asset (secret key) from normal application code, ensuring that even if application registers are compromised, the secret key remains protected. The privilege-level-based access control creates a clear boundary between secure and non-secure operations.
Solution Approach 2:
The patent introduces a privilege level mechanism as an intermediary between application code and the secret key. Instead of allowing direct access to the secret key or using complex hardware encryption modules, the system uses the existing privilege level structure to mediate access. The secret key in the special-purpose register can only be accessed when the processor is in a higher privilege mode, providing a simple yet effective access control mechanism.
3Reliability
If trap interrupt is triggered on hash mismatch, then security response to attacks is improved, but system performance decreases due to interrupt overhead
Solution Approach 1:
The patent implements preliminary anti-action by verifying hash values at function exit before allowing the function to return. This pre-verification prevents malicious code from successfully executing returned-to-addresses obtained through ROP or DOP attacks. The trap interrupt is triggered only when verification fails, meaning the system proactively prevents attacks rather than reacting to them, reducing the frequency of interrupt execution.
Solution Approach 2:
The patent converts the potential harm of trap interrupt overhead into a benefit by designing the verification mechanism to trigger interrupts only on attack detection. In normal operation, the hash verification passes and no interrupt is generated, so the performance overhead is minimal. When an attack is detected, the trap interrupt provides a security benefit by preventing the malicious operation, transforming what could be seen as a performance penalty into a security feature that only activates when needed.
Data Source
AI summary
In response to encountering a function entry instruction while running an application, a processor computes an entry hash value using a hash of three hash input parameters, wherein one of the input parameters is a secret key stored in the special purpose register. In response to encountering a function exit instruction, the processor computes an exit hash value using the same three input parameters and the same hash used when computing the entry hash value. The processor determines if the entry hash value is the same as the exit hash value. In response to the entry hash value being the same as the exit hash value the processor exits the function and in response to the entry hash value not being the same as the exit hash value the processor executes a trap interrupt.