Hooking Application Compatibility via Stub DLL and Vector

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing hooking applications fail to coexist on a system, leading to issues such as incorrect return addresses and invalid API invocations, which result in system failures when multiple hooking applications attempt to hook the same interface.

Innovation Solution

A system and method utilizing a vector and a stub dynamic link library (DLL) allow multiple hooking applications to coexist by managing return addresses and maintaining a persistent handler, enabling them to hook an interface simultaneously and handle situations like unloading and upgrading without system failure.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If multiple hooking applications attempt to hook the same interface, then the quantity of hooking applications increases, but system reliability deteriorates due to incorrect return addresses and invalid API invocations

Engineering Contradiction:
Improvequantity of hooking applicationsVSAvoidsystem reliability
Core Design Contradiction:
Quantity of substanceVSReliability

Solution Approach 1:

The patent introduces a hooking application compatibility layer as an intermediary component that sits between multiple hooking applications and the target interface. This compatibility layer manages return address storage and retrieval, ensuring that each hooking application receives the correct return address even when multiple applications are stacked. The intermediary prevents direct conflicts between hooking applications by mediating their interactions through a standardized interface that handles address management centrally.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If a second hooking application invokes the first hooking application, then the second application can access the first application's handler, but the return address points into the first hooking application instead of the original API, causing failure

Engineering Contradiction:
Improveability to invoke handlersVSAvoidreturn address validity
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The compatibility layer performs preliminary actions by pre-storing return addresses in a structured data structure before any hooking application attempts to invoke handlers. When a hooking application installs its hook, the compatibility layer captures and stores the original API's return address in advance. This preliminary storage ensures that even when applications are invoked in stacked configurations, the correct return address is already available and can be retrieved without pointing to invalid locations within other hooking applications.

Inventive Principle:
Principle #10Preliminary action

3Adaptability or versatility

If the first hooking application unloads, then the second hooking application cannot invoke the first application's handler, but without the compatibility layer, the system transfers control to an invalid address causing failure

Engineering Contradiction:
Improveability to unload applicationsVSAvoidaddress validity after unloading
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The compatibility layer creates and maintains copies of return address information in a persistent data structure that survives application unloading. Instead of relying on the actual presence of the first hooking application in memory, the compatibility layer stores copies of the necessary address information that can be retrieved even after the original application has unloaded. This copying mechanism ensures that the second hooking application can still function correctly by retrieving the stored return address from the compatibility layer's data structure, avoiding attempts to invoke handlers from unloaded applications.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS7779422B1System, method, and computer program product for compatibility among hooking applications
Publication Date: 2010.08.17 MCAFEE LLC
  • US7779422B1 patent drawing
  • US7779422B1 patent drawing
  • US7779422B1 patent drawing

AI summary

A system, method and computer program product for permitting use of multiple hooking applications to hook an interface. Included are a first hooking application and a second hooking application. Further provided is computer code including a vector and a stub dynamic link library (DLL) for permitting the first hooking application and the second hooking application to both hook an interface.