ROP Exploit Detection via Stack Address Comparison

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods, such as Data Execution Prevention (DEP) and Address Space Layout Randomization (ASLR), are ineffective against Return-Oriented Programming (ROP) exploits that evade DEP by using existing executable code, particularly in software applications that do not utilize ASLR.

Innovation Solution

A method is established to detect ROP exploitation by hooking code locations and comparing their addresses with values in the freed stack space, determining if a control transfer relates to ROP exploitation, and taking action to prevent it, specifically by checking if an electronic file does not use ASLR and estimating the number of location addresses to compare with the return address.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Object-affected harmful factors

If DEP is used to mark memory regions as non-executable, then code injection attacks are prevented, but ROP exploits can still bypass it by using existing executable code

Engineering Contradiction:
Improvecode injection attack preventionVSAvoidROP exploit vulnerability
Core Design Contradiction:
Object-affected harmful factorsVSReliability

Solution Approach 1:

The patent applies preliminary action by pre-establishing hooking rules at code locations before ROP exploits can execute. The system proactively sets up detection mechanisms that monitor control transfers and compare addresses with freed stack space values, enabling detection of ROP exploitation attempts before they can successfully bypass DEP protections.

Inventive Principle:
Principle #10Preliminary action

2Difficulty of detecting and measuring

If ASLR is used to randomly arrange memory locations, then ROP exploit prediction becomes difficult, but applications without ASLR remain vulnerable

Engineering Contradiction:
Improvememory address prediction difficultyVSAvoidapplication vulnerability
Core Design Contradiction:
Difficulty of detecting and measuringVSReliability

Solution Approach 1:

The patent applies universality by creating a detection mechanism that works across both ASLR-enabled and ASLR-disabled applications. The hooking rule system and address comparison method provide universal ROP exploitation detection capability, making the security solution applicable to all applications regardless of whether they implement ASLR.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Object-affected harmful factors

If ROP exploits use existing executable code instead of injecting malicious code, then DEP becomes ineffective, but the exploit can still execute malicious payloads

Engineering Contradiction:
ImproveDEP effectivenessVSAvoidmalicious code execution
Core Design Contradiction:
Object-affected harmful factorsVSObject-generated harmful factors

Solution Approach 1:

The patent applies the intermediary principle by introducing a detection layer that mediates between the executed code and the system. The hooking rules act as intermediaries that monitor control transfers and intercept potential ROP exploitation attempts by comparing addresses with freed stack space, preventing malicious code execution even when using existing executable code.

Inventive Principle:
Principle #24Intermediary (Mediator)

4Measurement precision

If the system monitors all control transfers to detect ROP exploitation, then detection accuracy improves, but system performance and complexity increase

Engineering Contradiction:
ImproveROP exploitation detection accuracyVSAvoiddetection system complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies local quality by focusing monitoring efforts on specific critical code locations rather than uniformly monitoring all code. Hooking rules are established at strategically important code locations where ROP exploitation is most likely to occur, and address comparison is performed specifically against freed stack space values, providing targeted detection with reduced overhead.

Inventive Principle:
Principle #3Local quality

Data Source

PatentUS9846779B2Detecting a return-oriented programming exploit
Publication Date: 2017.12.19 WITHSECURE CORP (A K A WITHSECURE OYJ)
  • US9846779B2 patent drawing
  • US9846779B2 patent drawing
  • US9846779B2 patent drawing

AI summary

A method and apparatus for detecting a Return-Oriented Programming exploitation. At a computer device, a mechanism to detect a control transfer of a code location in a memory is established. This may be, for example, hooking the control transfer. The code location relates to an electronic file. In the event that a control transfer of the code location is detected, a comparison is made between a destination code location address with values in the freed stack. If the code location address matches any of the values in the freed stack, then it is determined that the control transfer of the code location relates to a Return-Oriented Programming exploitation.