Subroutine Return Address Mutation for Binary Instrumentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods struggle to effectively instrument the return from a subroutine in binaries, as it is difficult to determine the correct return address due to compiled code complexities, such as multiple returns and interspersed read-only data, making it challenging to trap the exit from a subroutine during runtime.

Innovation Solution

The return address of a subroutine is mutated to become an invalid instruction pointer, triggering a fault and an exception handler, which instruments the exit, determines the correct return address, and restores it, allowing for dynamic instrumentation of the return from the subroutine without recompiling the program.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If code instrumentation is performed by placing additional instructions in the code, then monitoring capability is improved, but device complexity and difficulty of operation increase due to requiring source code access and recompilation

Engineering Contradiction:
Improvemonitoring capabilityVSAvoidease of instrumentation
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The patent creates a copy of the return address value and stores it in a safe location before mutation. This copying mechanism allows the original return address to be preserved for later restoration while enabling the mutation needed for trapping. The copy serves as a placeholder that maintains program correctness while allowing instrumentation intervention.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent introduces an intermediary exception handler that mediates between the mutated return address and the original return address. When the mutated return address triggers an exception, the handler restores the original return address from the copied value, enabling monitoring without permanently altering program execution flow. This intermediary layer facilitates instrumentation while maintaining binary compatibility.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If the return address is mutated to trigger an exception handler for instrumentation, then the ability to trap subroutine exits is improved, but reliability risks increase due to potential program crashes from invalid instruction pointers

Engineering Contradiction:
Improvetrapping capabilityVSAvoidprogram stability
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent performs preliminary action by copying the return address value before mutation occurs. This pre-copying ensures that the original valid return address is preserved in a safe location before being transformed into an invalid instruction pointer. The preliminary preservation of the return address value prevents program crashes by ensuring restoration capability is already in place before the mutation takes effect.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent provides beforehand cushioning by storing a copy of the original return address in a protected location before mutation. This cushioning mechanism acts as a safety buffer that prevents the program from crashing when the mutated return address is encountered. The pre-stored copy serves as a protective measure that absorbs the potential harm of the mutation and enables safe recovery.

Inventive Principle:
Principle #11Beforehand cushioning (Prior cushioning)

3Adaptability or versatility

If instrumentation is performed on compiled binaries without source code access, then adaptability is improved, but manufacturing precision deteriorates due to inability to precisely identify return addresses in compiled code

Engineering Contradiction:
Improvebinary compatibilityVSAvoidreturn address identification accuracy
Core Design Contradiction:
Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent employs self-service by having the program itself provide the return address information through its own exception handling mechanism. Instead of requiring external analysis tools to identify return addresses in compiled code, the system leverages the program's built-in exception handling infrastructure. The mutation of the return address triggers the program's own exception handler, which then restores the address, enabling precise identification and instrumentation without external assistance.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent achieves universality by making the instrumentation technique applicable to any subroutine with a return address, regardless of how it was compiled or what language it was written in. The method works with the universal exception handling mechanism present in most modern systems, making it language-independent and compiler-independent. This multi-functionality allows the same technique to instrument application code, system libraries, and kernel routines uniformly.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS9129062B1Intercepting subroutine return in unmodified binaries
Publication Date: 2015.09.08 VMWARE INC
  • US9129062B1 patent drawing
  • US9129062B1 patent drawing
  • US9129062B1 patent drawing

AI summary

Systems and methods for instrumenting code are disclosed. The entry to a subroutine is trapped and the subroutine's return address is mutated to create an invalid instruction pointer. The mutated return address is stored in the architecture reserved space for the return address. An exception handler is executed that has been instrumented to handle the fault caused by the mutated return address such that the exit from the subroutine is instrumented.